Skip to content

Add PythonEnvInfo-related helpers. #14051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
1126834
Drop `PythonEnvInfo.id`.
ericsnowcurrently Sep 22, 2020
0d22681
Make buildEmptyEnvInfo() a public helper.
ericsnowcurrently Sep 23, 2020
4468e74
Add a copyEnvInfo() helper.
ericsnowcurrently Sep 23, 2020
94137c8
Add locator utils.
ericsnowcurrently Sep 21, 2020
eea499b
Use getEnvs() in ComponentAdapter.getInterpreters().
ericsnowcurrently Sep 23, 2020
7a2fdfc
Add PythonEnvUpdatedEvent.index.
ericsnowcurrently Sep 23, 2020
ca2ec73
Fix reducer and resolver.
ericsnowcurrently Sep 28, 2020
af6c7c5
Make parseVersion() a public helper.
ericsnowcurrently Sep 23, 2020
e737607
buildEmptyEnvInfo() -> buildEnvInfo().
ericsnowcurrently Sep 23, 2020
51c6dee
Factor out updateEnv().
ericsnowcurrently Sep 23, 2020
2ecd537
Use lodash.cloneDeep() to deep-copy objects.
ericsnowcurrently Sep 23, 2020
9cd6c7d
Drop the testing helper createEnv().
ericsnowcurrently Sep 23, 2020
6b50973
Factor out getURIFilter().
ericsnowcurrently Sep 24, 2020
ad8f195
Allow locator queries to mix search locations and non.
ericsnowcurrently Sep 24, 2020
2b70a1c
Match subdirectories in locator queries.
ericsnowcurrently Sep 24, 2020
7b19538
Make PythonVersion.release optional.
ericsnowcurrently Sep 24, 2020
eb746f9
Drop the fixPath() testing helper.
ericsnowcurrently Sep 24, 2020
20c46da
lint
ericsnowcurrently Sep 24, 2020
9d970f5
Clarify what a missing query field means.
ericsnowcurrently Sep 28, 2020
ce53e54
Make PythonLocatorQuery.searchLocations more explicit about the non-r…
ericsnowcurrently Sep 28, 2020
3d85fa7
Fix a sonarcloud warning.
ericsnowcurrently Sep 28, 2020
e47322b
Filter out undefined locations in the tests.
ericsnowcurrently Sep 28, 2020
f97bb31
Simplify the searchLocations logic a little.
ericsnowcurrently Sep 28, 2020
1edb76d
Strip trailing slashes in paths.
ericsnowcurrently Sep 28, 2020
fd5e37b
Add getMinimalPartialInfo().
ericsnowcurrently Sep 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions src/client/common/utils/misc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,53 @@ export function isUri(resource?: Uri | any): resource is Uri {
return typeof uri.path === 'string' && typeof uri.scheme === 'string';
}

/**
* Create a filter func that determine if the given URI and candidate match.
*
* The scheme must match, as well as path.
*
* @param checkParent - if `true`, match if the candidate is rooted under `uri`
* @param checkChild - if `true`, match if `uri` is rooted under the candidate
* @param checkExact - if `true`, match if the candidate matches `uri` exactly
*/
export function getURIFilter(
uri: Uri,
opts: {
checkParent?: boolean;
checkChild?: boolean;
checkExact?: boolean;
} = { checkExact: true }
): (u: Uri) => boolean {
let uriPath = uri.path;
while (uri.path.endsWith('/')) {
uriPath = uriPath.slice(0, -1);
}
const uriRoot = `${uriPath}/`;
function filter(candidate: Uri): boolean {
if (candidate.scheme !== uri.scheme) {
return false;
}
let candidatePath = candidate.path;
while (candidate.path.endsWith('/')) {
candidatePath = candidatePath.slice(0, -1);
}
if (opts.checkExact && candidatePath === uriPath) {
return true;
}
if (opts.checkParent && candidatePath.startsWith(uriRoot)) {
return true;
}
if (opts.checkChild) {
const candidateRoot = `{candidatePath}/`;
if (uriPath.startsWith(candidateRoot)) {
return true;
}
}
return false;
}
return filter;
}

export function isNotebookCell(documentOrUri: TextDocument | Uri): boolean {
const uri = isUri(documentOrUri) ? documentOrUri : documentOrUri.uri;
return uri.scheme.includes(NotebookCellScheme);
Expand Down
139 changes: 127 additions & 12 deletions src/client/pythonEnvironments/base/info/env.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,123 @@

import { cloneDeep } from 'lodash';
import * as path from 'path';
import { Architecture } from '../../../common/utils/platform';
import { arePathsSame } from '../../common/externalDependencies';
import { areEqualVersions, areEquivalentVersions } from './pythonVersion';

import {
FileInfo,
PythonDistroInfo,
PythonEnvInfo, PythonEnvKind, PythonVersion,
PythonEnvInfo,
PythonEnvKind,
PythonReleaseLevel,
PythonVersion,
} from '.';
import { Architecture } from '../../../common/utils/platform';
import { arePathsSame } from '../../common/externalDependencies';
import { areEqualVersions, areEquivalentVersions } from './pythonVersion';

/**
* Create a new info object with all values empty.
*
* @param init - if provided, these values are applied to the new object
*/
export function buildEnvInfo(init?: {
kind?: PythonEnvKind;
executable?: string;
location?: string;
version?: PythonVersion;
}): PythonEnvInfo {
const env = {
kind: PythonEnvKind.Unknown,
executable: {
filename: '',
sysPrefix: '',
ctime: -1,
mtime: -1,
},
name: '',
location: '',
version: {
major: -1,
minor: -1,
micro: -1,
release: {
level: PythonReleaseLevel.Final,
serial: 0,
},
},
arch: Architecture.Unknown,
distro: {
org: '',
},
};
if (init !== undefined) {
updateEnv(env, init);
}
return env;
}

/**
* Return a deep copy of the given env info.
*
* @param updates - if provided, these values are applied to the copy
*/
export function copyEnvInfo(
env: PythonEnvInfo,
updates?: {
kind?: PythonEnvKind,
},
): PythonEnvInfo {
// We don't care whether or not extra/hidden properties
// get preserved, so we do the easy thing here.
const copied = cloneDeep(env);
if (updates !== undefined) {
updateEnv(copied, updates);
}
return copied;
}

function updateEnv(env: PythonEnvInfo, updates: {
kind?: PythonEnvKind;
executable?: string;
location?: string;
version?: PythonVersion;
}): void {
if (updates.kind !== undefined) {
env.kind = updates.kind;
}
if (updates.executable !== undefined) {
env.executable.filename = updates.executable;
}
if (updates.location !== undefined) {
env.location = updates.location;
}
if (updates.version !== undefined) {
env.version = updates.version;
}
}

/**
* For the given data, build a normalized partial info object.
*
* If insufficient data is provided to generate a minimal object, such
* that it is not identifiable, then `undefined` is returned.
*/
export function getMinimalPartialInfo(env: string | Partial<PythonEnvInfo>): Partial<PythonEnvInfo> | undefined {
if (typeof env === 'string') {
if (env === '') {
return undefined;
}
return {
executable: { filename: env, sysPrefix: '', ctime: -1, mtime: -1 },
};
}
if (env.executable === undefined) {
return undefined;
}
if (env.executable.filename === '') {
return undefined;
}
return env;
}

/**
* Checks if two environments are same.
Expand All @@ -24,14 +133,20 @@ import { areEqualVersions, areEquivalentVersions } from './pythonVersion';
* to be same environment. This later case is needed for comparing windows store python,
* where multiple versions of python executables are all put in the same directory.
*/
export function areSameEnvironment(
left: string | PythonEnvInfo,
right: string | PythonEnvInfo,
export function areSameEnv(
left: string | Partial<PythonEnvInfo>,
right: string | Partial<PythonEnvInfo>,
allowPartialMatch?: boolean,
): boolean {
const leftFilename = typeof left === 'string' ? left : left.executable.filename;
const rightFilename = typeof right === 'string' ? right : right.executable.filename;
): boolean | undefined {
const leftInfo = getMinimalPartialInfo(left);
const rightInfo = getMinimalPartialInfo(right);
if (leftInfo === undefined || rightInfo === undefined) {
return undefined;
}
const leftFilename = leftInfo.executable!.filename;
const rightFilename = rightInfo.executable!.filename;

// For now we assume that matching executable means they are the same.
if (arePathsSame(leftFilename, rightFilename)) {
return true;
}
Expand Down Expand Up @@ -72,11 +187,11 @@ function getPythonVersionInfoHeuristic(version:PythonVersion): number {
infoLevel += 5; // W2
}

if (version.release.level) {
if (version.release?.level) {
infoLevel += 3; // W1
}

if (version.release.serial || version.sysVersion) {
if (version.release?.serial || version.sysVersion) {
infoLevel += 1; // W0
}

Expand Down
13 changes: 6 additions & 7 deletions src/client/pythonEnvironments/base/info/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ export enum PythonEnvKind {
OtherVirtual = 'virt-other'
}

/**
* A (system-global) unique ID for a single Python environment.
*/
export type PythonEnvID = string;

/**
* Information about a file.
*/
Expand All @@ -44,11 +49,6 @@ export type PythonExecutableInfo = FileInfo & {
sysPrefix: string;
};

/**
* A (system-global) unique ID for a single Python environment.
*/
export type PythonEnvID = string;

/**
* The most fundamental information about a Python environment.
*
Expand All @@ -63,7 +63,6 @@ export type PythonEnvID = string;
* @prop location - the env's location (on disk), if relevant
*/
export type PythonEnvBaseInfo = {
id: PythonEnvID;
kind: PythonEnvKind;
executable: PythonExecutableInfo;
// One of (name, location) must be non-empty.
Expand Down Expand Up @@ -99,7 +98,7 @@ export type PythonVersionRelease = {
* @prop sysVersion - the raw text from `sys.version`
*/
export type PythonVersion = BasicVersionInfo & {
release: PythonVersionRelease;
release?: PythonVersionRelease;
sysVersion?: string;
};

Expand Down
6 changes: 5 additions & 1 deletion src/client/pythonEnvironments/base/info/pythonVersion.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

import { PythonReleaseLevel, PythonVersion } from '.';
import { EMPTY_VERSION, parseBasicVersionInfo } from '../../../common/utils/version';

import { PythonReleaseLevel, PythonVersion } from '.';

/**
* Convert the given string into the corresponding Python version object.
*/
export function parseVersion(versionStr: string): PythonVersion {
const parsed = parseBasicVersionInfo<PythonVersion>(versionStr);
if (!parsed) {
Expand Down
37 changes: 27 additions & 10 deletions src/client/pythonEnvironments/base/locator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,17 +15,18 @@ import {
* A single update to a previously provided Python env object.
*/
export type PythonEnvUpdatedEvent = {
/**
* The iteration index of The env info that was previously provided.
*/
index: number;
/**
* The env info that was previously provided.
*
* If the event comes from `IPythonEnvsIterator.onUpdated` then
* `old` was previously yielded during iteration.
*/
old: PythonEnvInfo;
old?: PythonEnvInfo;
Copy link

@karrtikr karrtikr Sep 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW I don't think we neeed old at all. We normally use this to search for the environment which is to be replaced by update, but we can also use update to search for the environment to replace.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason to keep it is because the object may not be the same as the one at the index. The one at the index is the one to use as "old", but it can be helpful to have the event.old as well, especially when debugging. So I'd like to keep it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason to keep it is because the object may not be the same as the one at the index

Maybe I don't understand your point, but we're using areSameEnvironments to look for matching environments, so we do not need to do exact object comparison to replace the one at the index. The only purpose for event.old was to search for the old environment which was fired and replace it.

Also, in the resolver, I found out that it's a bit hard to maintain the exact "old" environment to be used for firing in certain circumstances.

but it can be helpful to have the event.old as well, especially when debugging

Yaa maybe, although if that's the only reason I think we should remove it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is event.old only for debugging purposes?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing old would take some work, so I'm fine with keeping it for now. I was just curious about the above question^^

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could certainly be used for debugging. However, I can imagine it being used for other things too.

/**
* The env info that replaces the old info.
*/
new: PythonEnvInfo;
update: PythonEnvInfo;
};

/**
Expand Down Expand Up @@ -73,23 +74,39 @@ export const NOOP_ITERATOR: IPythonEnvsIterator = iterEmpty<PythonEnvInfo>();
* This is directly correlated with the `BasicPythonEnvsChangedEvent`
* emitted by watchers.
*
* @prop kinds - if provided, results should be limited to these env kinds
* @prop kinds - if provided, results should be limited to these env
* kinds; if not provided, the kind of each evnironment
* is not considered when filtering
*/
export type BasicPythonLocatorQuery = {
kinds?: PythonEnvKind[];
};

/**
* The portion of a query related to env search locations.
*/
export type SearchLocations = {
/**
* The locations under which to look for environments.
*/
roots: Uri[];
/**
* If true, also look for environments that do not have a search location.
*/
includeNonRooted?: boolean;
};

/**
* The full set of possible info to send to a locator when requesting environments.
*
* This is directly correlated with the `PythonEnvsChangedEvent`
* emitted by watchers.
*
* @prop - searchLocations - if provided, results should be limited to
* within these locations
*/
export type PythonLocatorQuery = BasicPythonLocatorQuery & {
searchLocations?: Uri[];
/**
* If provided, results should be limited to within these locations.
*/
searchLocations?: SearchLocations;
};

type QueryForEvent<E> = E extends PythonEnvsChangedEvent ? PythonLocatorQuery : BasicPythonLocatorQuery;
Expand Down
Loading