Skip to content

Production decomposition #469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 98 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
c09cf80
Add configuration setting for production decomposition
isc-pbarton May 7, 2024
0f58789
Export decomposed production after save
isc-pbarton May 14, 2024
10b6a27
External name to internal name logic for PTD - most of this should be…
isc-pbarton May 17, 2024
be0f76f
Added default mapping for decomposed production items
isc-pbarton Jul 9, 2024
88b1617
work in progress - only export modified production items
isc-pbarton Jul 9, 2024
aef8112
Export decomposed production after save
isc-pbarton May 14, 2024
273568c
remove method that got duplicated by bad resolve
isc-pbarton Jul 12, 2024
8c07923
Merge remote-tracking branch 'origin' into production-change-control
isc-pbarton Jul 12, 2024
d365e82
Merge branch 'main' into production-change-control
isc-pbarton Jul 18, 2024
cd8eea0
fix for add to source control because PTD internal name doesn't exist…
isc-pbarton Jul 18, 2024
6885262
more fixes to name logic for PTDs
isc-pbarton Jul 18, 2024
383bdae
GetStatus logic to prohibit editing item checked out by other user
isc-pbarton Jul 18, 2024
d5aaf6c
useful alert shown to user; remove spurious alert on loading config p…
isc-pbarton Jul 18, 2024
75d4ca0
import PTD uses deployment manager to apply setting
isc-pbarton Jul 18, 2024
5958615
Refactored naming logic into production class
isc-pbarton Jul 23, 2024
4cca397
Merge branch 'main' into production-change-control
isc-pbarton Jul 23, 2024
97364de
remove unused argument on parseexternal/internalname
isc-pbarton Jul 24, 2024
d54b0e2
import all now imports user types in subdirectories recursively
isc-pbarton Jul 25, 2024
3733eb5
import production class without definition
isc-pbarton Jul 25, 2024
67b1995
refactor: better way of determining if namespace has Ensemble
isc-pbarton Jul 25, 2024
47b25c7
fix deleting production items deletes file from source control
isc-pbarton Jul 25, 2024
afb7e34
fix issue where importing PTD items reexports them as changed
isc-pbarton Jul 26, 2024
8e76844
Production classes now excluded from export/import
isc-pbarton Aug 6, 2024
85bb105
Add configuration setting for production decomposition
isc-pbarton May 7, 2024
5579de7
Export decomposed production after save
isc-pbarton May 14, 2024
f3bd74d
External name to internal name logic for PTD - most of this should be…
isc-pbarton May 17, 2024
de56a87
Added default mapping for decomposed production items
isc-pbarton Jul 9, 2024
6c679c2
work in progress - only export modified production items
isc-pbarton Jul 9, 2024
771d178
Export decomposed production after save
isc-pbarton May 14, 2024
5225402
remove method that got duplicated by bad resolve
isc-pbarton Jul 12, 2024
82d1836
fix for add to source control because PTD internal name doesn't exist…
isc-pbarton Jul 18, 2024
48ceba9
more fixes to name logic for PTDs
isc-pbarton Jul 18, 2024
ab404ff
GetStatus logic to prohibit editing item checked out by other user
isc-pbarton Jul 18, 2024
a74b72a
useful alert shown to user; remove spurious alert on loading config p…
isc-pbarton Jul 18, 2024
4ae9fcd
import PTD uses deployment manager to apply setting
isc-pbarton Jul 18, 2024
e9fd878
Refactored naming logic into production class
isc-pbarton Jul 23, 2024
7bf670f
remove unused argument on parseexternal/internalname
isc-pbarton Jul 24, 2024
55d0986
import all now imports user types in subdirectories recursively
isc-pbarton Jul 25, 2024
cb1c167
import production class without definition
isc-pbarton Jul 25, 2024
4225957
refactor: better way of determining if namespace has Ensemble
isc-pbarton Jul 25, 2024
e507af0
fix deleting production items deletes file from source control
isc-pbarton Jul 25, 2024
6bb25c3
fix issue where importing PTD items reexports them as changed
isc-pbarton Jul 26, 2024
5ab95d5
Production classes now excluded from export/import
isc-pbarton Aug 6, 2024
28c98d1
Merge branch 'production-change-control' of https://github.com/inters…
isc-pbarton Aug 6, 2024
6e16c7d
production utilities moved to class in git-source-control
isc-pbarton Aug 20, 2024
6ad0ab4
Merge branch 'main' into production-change-control
isc-pbarton Aug 20, 2024
7b010c5
Merge branch 'main' into production-change-control
isc-pbarton Aug 26, 2024
098a708
fix: production items are exported for source control when production…
isc-pbarton Aug 26, 2024
01d0759
deletes of PTD items now delete associated config item from production
isc-pbarton Aug 27, 2024
9024dac
fix: git pull now successfully deletes production config items
isc-pbarton Aug 27, 2024
1c3bb4d
port CreateProduction from CCR
isc-pbarton Aug 27, 2024
dd221fc
deploying production items will create production class if it does no…
isc-pbarton Aug 27, 2024
2398ac0
Revert "fix: git pull now successfully deletes production config items"
isc-pbarton Aug 30, 2024
0c469ba
deployment deletes production items
isc-pbarton Sep 4, 2024
4605a98
fix: stale ptd entries in items cache no longer breaks Export All
isc-pbarton Sep 4, 2024
0528427
added production baseline to be run after enabling or disabling produ…
isc-pbarton Sep 4, 2024
e396675
fix: items cache now populated correctly when adding new production
isc-pbarton Sep 6, 2024
f0652de
added tests for production decomposition
isc-pbarton Sep 6, 2024
15a2760
production decomposition refactoring and code style improvements
isc-pbarton Sep 6, 2024
a86e8b8
settings UI for decompose productions flag
isc-pbarton Sep 9, 2024
dab1a6f
fix bug getting modified production item settings
isc-pbarton Sep 10, 2024
5778f7f
production decomp unit test correctly tests deployment
isc-pbarton Sep 10, 2024
43c4b71
git-source-control settings are now a source-controlled file
isc-pbarton Sep 17, 2024
c876c9f
Merge branch 'main' into production-change-control
isc-pbarton Sep 17, 2024
0557e04
Merge branch 'main' into production-change-control
isc-pbarton Sep 24, 2024
9fb2104
docs: documentation and testing plan for production decomposition
isc-pbarton Sep 27, 2024
4ea1312
enabling production decomposition creates a default mapping for PTD i…
isc-pbarton Sep 27, 2024
fc3c599
Merge remote-tracking branch 'origin/main' into production-change-con…
isc-pbarton Sep 27, 2024
fc8aa96
fix issues with settings file on repo initialization
isc-pbarton Sep 30, 2024
83f12ba
Merge branch 'main' into production-change-control
isc-pbarton Oct 1, 2024
53c5985
fix: settings import/export supports NoFolders
isc-pbarton Oct 1, 2024
c0ae6bd
Merge remote-tracking branch 'origin/main' into production-change-con…
isc-pbarton Oct 2, 2024
d04fa4e
Port production changes from CCR
isc-pbarton Oct 2, 2024
7fa1872
fix: privilege violation with getting modified production items
isc-pbarton Oct 2, 2024
aaa1336
docs: standardize casing of Production Decomposition feature name
isc-pbarton Oct 4, 2024
6cd7b3b
fix: production decomp SQL privilege violations and backwards incompa…
isc-pbarton Oct 4, 2024
527745a
Merge branch 'main' into production-change-control
isc-pbarton Oct 4, 2024
1cefb56
Merge branch 'main' into production-change-control
isc-pbarton Oct 9, 2024
7df8d2c
fix: issue importing production items when namespace has multiple pro…
isc-pbarton Oct 16, 2024
3a375f0
enh: partial support for editing decomposed production from IDE
isc-pbarton Oct 16, 2024
201ccd7
remove last modified timestamp from PTD files after export
isc-pbarton Oct 17, 2024
c34fa58
by default, lock editing of decomposed productions in the IDE
isc-pbarton Oct 18, 2024
e6479a8
Merge branch 'main' into production-change-control
isc-pbarton Oct 18, 2024
b77b025
fix issue detecting production edits from VS Code
isc-pbarton Oct 18, 2024
83ffd8e
update changelog for production decomposition
isc-pbarton Oct 18, 2024
b22e84b
Merge branch 'main' into production-change-control
isc-pbarton Oct 29, 2024
ebc44c0
Merge branch 'main' into production-change-control
isc-pbarton Oct 30, 2024
ed19c80
fix: PTD items should not be compiled
isc-pbarton Oct 30, 2024
bd27d4a
fix: remove duplicate success message on settings save
isc-pbarton Oct 31, 2024
d784f25
Merge branch 'main' into production-change-control
isc-pbarton Nov 5, 2024
e84bd10
chore: update version string for next version
isc-pbarton Nov 5, 2024
6305cc4
fix: production items are deployed after all changed classes are comp…
isc-pbarton Nov 5, 2024
1f7dd47
fix: fixes and updates to embedded git config file
isc-pbarton Nov 12, 2024
391fd88
port production class changes from %Studio.SourceControl
isc-pbarton Nov 12, 2024
ad30a5e
fix: PTD items may be deployed into locked namespace
isc-pbarton Nov 12, 2024
c6e7c5e
Merge branch 'main' of https://github.com/intersystems/git-source-con…
isc-pbarton Nov 12, 2024
b5ca7ae
Merge branch 'main' into production-change-control
isc-tleavitt Nov 15, 2024
e8e7209
chore: remove srcver ported from client tools
isc-pbarton Nov 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.8.0] - Unreleased

### Added
- Production Decomposition mode allows controlling interoperability productions as individual files for each host (#469)

## [2.7.1] - 2024-11-13

### Fixed
Expand Down
7 changes: 7 additions & 0 deletions cls/SourceControl/Git/API.cls
Original file line number Diff line number Diff line change
Expand Up @@ -75,4 +75,11 @@ ClassMethod MapEverywhere()
Quit ##class(SourceControl.Git.Installer).MapEverywhere()
}

/// Run to baseline all interoperability productions in the namespace to source control.
/// This should be done after changing the value of the "decompose productions" setting.
ClassMethod BaselineProductions()
{
do ##class(SourceControl.Git.Util.Production).BaselineProductions()
}

}
82 changes: 68 additions & 14 deletions cls/SourceControl/Git/Extension.cls
Original file line number Diff line number Diff line change
Expand Up @@ -327,22 +327,44 @@ Method OnBeforeTimestamp(InternalName As %String)
Method OnAfterSave(InternalName As %String, Object As %RegisteredObject = {$$$NULLOREF}) As %Status
{
set sc = $$$OK
quit:$get(%gscSkipSaveHooks) sc
try {
set InternalName = ##class(SourceControl.Git.Utils).NormalizeInternalName(.InternalName,.fromWebApp,.fullExternalName)
set context = ##class(SourceControl.Git.PackageManagerContext).ForInternalName(InternalName)
if ##class(SourceControl.Git.Utils).IsNamespaceInGit() && ..IsInSourceControl(InternalName) {
if fromWebApp {
if fullExternalName = ##class(SourceControl.Git.Utils).FullExternalName(InternalName) {
// Reimport item into database
$$$ThrowOnError(##class(SourceControl.Git.Utils).ImportItem(InternalName,,1,1))
if ##class(SourceControl.Git.Utils).IsNamespaceInGit() {
// If this is a production class and production decomposition is enabled, call recursively on all modified production items.
if ##class(SourceControl.Git.Utils).ItemIsProductionToDecompose(InternalName) {
do ##class(SourceControl.Git.Production).GetModifiedItemsAfterSave(InternalName, .productionItems)
set key = $order(productionItems(""))
while (key '= "") {
if productionItems(key) = "D" {
set itemFilename = ..FullExternalName(key)
if ##class(SourceControl.Git.Utils).IsInSourceControl(key) && ##class(%File).Exists(itemFilename) {
$$$ThrowOnError(##class(SourceControl.Git.Change).AddDeletedToUncommitted(itemFilename, key))
$$$ThrowOnError(##class(SourceControl.Git.Utils).DeleteExternalFile(key))
}
} elseif '..IsInSourceControl(key) {
$$$ThrowOnError(##class(SourceControl.Git.Utils).AddToSourceControl(key))
} else {
$$$ThrowOnError(..OnAfterSave(key))
}
set key = $order(productionItems(key))
}
} else {
set filename = ##class(SourceControl.Git.Utils).FullExternalName(InternalName)
$$$ThrowOnError(##class(SourceControl.Git.Utils).RemoveRoutineTSH(InternalName))
set forceExport = (InternalName'= "") && ($data(..Modified(InternalName)))
$$$ThrowOnError(##class(SourceControl.Git.Utils).ExportItem(InternalName,,forceExport))
if '##class(SourceControl.Git.Change).IsUncommitted(filename) {
$$$ThrowOnError(##class(SourceControl.Git.Change).SetUncommitted(filename, "edit", InternalName, $username, "", 1, "", "", 0))
}
if ..IsInSourceControl(InternalName) {
if fromWebApp {
if fullExternalName = ##class(SourceControl.Git.Utils).FullExternalName(InternalName) {
// Reimport item into database
$$$ThrowOnError(##class(SourceControl.Git.Utils).ImportItem(InternalName,,1,1))
}
} else {
set filename = ##class(SourceControl.Git.Utils).FullExternalName(InternalName)
$$$ThrowOnError(##class(SourceControl.Git.Utils).RemoveRoutineTSH(InternalName))
set forceExport = (InternalName'= "") && ($data(..Modified(InternalName)))
$$$ThrowOnError(##class(SourceControl.Git.Utils).ExportItem(InternalName,,forceExport))
if '##class(SourceControl.Git.Change).IsUncommitted(filename) {
$$$ThrowOnError(##class(SourceControl.Git.Change).SetUncommitted(filename, "edit", InternalName, $username, "", 1, "", "", 0))
}
}
}
}
Expand Down Expand Up @@ -400,9 +422,41 @@ Method ExternalName(InternalName As %String) As %String
quit ##class(SourceControl.Git.Utils).ExternalName(InternalName)
}

ClassMethod FullExternalName(InternalName As %String) As %String
{
quit ##class(SourceControl.Git.Utils).FullExternalName(InternalName)
}

Method IsReadOnly(InternalName As %String) As %Boolean
{
quit ##class(SourceControl.Git.Utils).Locked()
set settings = ##class(SourceControl.Git.Settings).%New()
quit (##class(SourceControl.Git.Utils).Locked()
&& '$get(^IRIS.Temp("sscProd",$job,"bypassLock")))
|| (##class(SourceControl.Git.Utils).ItemIsProductionToDecompose($get(InternalName))
&& 'settings.decomposeProdAllowIDE
&& '##class(SourceControl.Git.Production).IsEnsPortal())
}

/// Called before the item is saved to the database it is passed
/// a reference to the current temporary storage of this item so that it
/// can be modified before the save completes. If you quit with an error
/// value then it will abort the save.
Method OnBeforeSave(InternalName As %String, Location As %String = "", Object As %RegisteredObject = {$$$NULLOREF}) As %Status
{
set st = $$$OK
if ##class(SourceControl.Git.Utils).ItemIsProductionToDecompose(InternalName) {
do ##class(SourceControl.Git.Production).GetModifiedItemsBeforeSave(InternalName,,.productionItems)
set key = $order(productionItems(""))
while (key '= "") {
// if any modified items in this production class are checked out by a different user, fail the check.
set st = ..GetStatus(key, .IsInSourceControl, .Editable, .IsCheckedOut, .UserCheckedOut)
quit:$$$ISERR(st)
if 'Editable set st = $$$ERROR($$$GeneralError,"Item is checked out by another user: "_UserCheckedOut)
quit:$$$ISERR(st)
set key = $order(productionItems(key))
}
}
return st
}

/// Check the status of the given item
Expand All @@ -411,7 +465,7 @@ Method IsReadOnly(InternalName As %String) As %Boolean
Method GetStatus(ByRef InternalName As %String, ByRef IsInSourceControl As %Boolean, ByRef Editable As %Boolean, ByRef IsCheckedOut As %Boolean, ByRef UserCheckedOut As %String) As %Status
{
set context = ##class(SourceControl.Git.PackageManagerContext).ForInternalName(.InternalName)
set Editable='..IsReadOnly(),IsCheckedOut=1,UserCheckedOut=""
set Editable='..IsReadOnly($get(InternalName)),IsCheckedOut=1,UserCheckedOut=""
set filename=##class(SourceControl.Git.Utils).FullExternalName(.InternalName)
set IsInSourceControl=(filename'=""&&($$$FileExists(filename)))
if filename="" quit $$$OK
Expand Down
19 changes: 17 additions & 2 deletions cls/SourceControl/Git/File.cls
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,23 @@ ClassMethod ExternalNameToInternalName(ExternalName As %String) As %String
}
new %SourceControl //don't trigger source hooks with this test load to get the Name
set sc=$system.OBJ.Load(ExternalName,"-d",,.outName,1)
if (($data(outName)=1) || ($data(outName) = 11 && ($order(outName(""),-1) = $order(outName(""))))) && ($zconvert(##class(SourceControl.Git.Utils).Type(outName),"U") '= "CSP") {
set itemIsPTD = 0
if $data(outName) = 11 {
set key = $order(outName(""))
while (key '= "") {
if ($zconvert($piece(outName,".",*),"U") = "PTD") {
set itemIsPTD = 1
quit
}
set key = $order(outName(key))
}
}
if itemIsPTD && ##class(%Library.EnsembleMgr).IsEnsembleNamespace() {
do ##class(SourceControl.Git.Production).ParseExternalName($replace(ExternalName,"\","/"),.internalName)
} elseif (($data(outName)=1) || ($data(outName) = 11 && ($order(outName(""),-1) = $order(outName(""))))) && ($zconvert(##class(SourceControl.Git.Utils).Type(outName),"U") '= "CSP") {
set internalName = outName
}
if (internalName '= "") {
set inst.InternalName = internalName
$$$ThrowOnError(inst.%Save())
}
Expand Down Expand Up @@ -68,4 +83,4 @@ Storage Default
<Type>%Storage.Persistent</Type>
}

}
}
6 changes: 5 additions & 1 deletion cls/SourceControl/Git/PackageManagerContext.cls
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ Method InternalNameSet(InternalName As %String = "") As %Status
set InternalName = ##class(SourceControl.Git.Utils).NormalizeInternalName(InternalName)
if (InternalName '= i%InternalName) {
set i%InternalName = InternalName
if (InternalName = ##class(SourceControl.Git.Settings.Document).#INTERNALNAME) {
// git source control settings document is never in an IPM context
quit $$$OK
}
if $$$comClassDefined("%IPM.ExtensionBase.Utils") {
set ..Package = ##class(%IPM.ExtensionBase.Utils).FindHomeModule(InternalName,,.resourceReference)
} elseif $$$comClassDefined("%ZPM.PackageManager.Developer.Extension.Utils") {
Expand Down Expand Up @@ -50,4 +54,4 @@ Method Dump()
write !?4,"Git-enabled? ",$select(..IsInGitEnabledPackage:"Yes",1:"No"),!
}

}
}
Loading
Loading