state of C# scripting in 2022?

[benblo]

I've been using cs-script off and on for quite a few years now, and recently I've gotten quite confused with all the options on the market: cs-script, dotnet-script, Roslyn/CLI/REPL mumbo-jumbo, scriptcs (OMG they just took your name and reversed it, how much more confusing can you get?!)... would you mind summing up their pros and cons?

Basically what I'm really wondering is, is dotnet-script the new standard, and is there still a point to having users install cs-script, when dotnet-script is more or less builtin now? Does cs-script allow for more advances usecases, eg Nuget directives?

[jasonswearingen]

I have just started researching this topic, literally this morning. Searching on the internet (google/ddg) is useless, swamped with useless stuff.

I found both cs-script and dotnet-script by searching on nuget. Both look very interesting and probably what I need (runtime scripting, letting the user type in some expression that gets evaluated in realtime)

However dotnet-script seems to have no documentation/samples ?!?!? also no documentation for Roslyn as far as I can see.

So for it's onboarding, seems like cs-script is the one I'll try out.

[oleg-shilo]

LOL
Mate, it's like asking Coca-Cola CEO which soft drink is better 😄

On a serious note, I feel funny about all this. CS-Script was my response to MS refusal to even consider support for executing C# in a scripting manner. in 2004 I got from a blunt response that "with .NET you do not need it". And this is when Python was already marching through the whole industry. And now... just looking at some of the scripting solutions with mixed feelings... seeing dotnet-script having almost all cs-script core features repeated and scriptcs even repeated it in the name. 😄
My greatest joy is that no one is debating if we need scripting.

I touched on some of these points in my interview for InfoQ, a long time ago.

TBH most I am not the best person to ask about other scripting products. Also, most likely you will find in every product some excellent features that are not present in others. Even though the features layout is dynamic and scripting solutions are constantly evolving and improving.

Thus I will simply focus on core CS-Script features and its "attitude".

CS-Script

• is the first and arguably the most mature scripting platform for .NET
• has Python dev-experience as an inspiration. Multiple file scripts (imports), packages, caching, hosting...
• uses plain vanilla ECMA-compliant C#. You can include a script file in the CV project and it will compile just fine.
• supports NuGet packages out of the box
• the same engine is used to execute scripts standalone (e.g. shell, cmd-prompt) and from the host applications (e.g. plugins).
• integration with OS.
  • singel global launcher css <script file> (css is a shim for the cs-script engine file)
  • infrastructure and temp files are not obstructing your dev-experience - a single script is all you need.
  • Installation from Chocolatey: choco install cs-script
  • Shell integration with Win and Linux
• Targets .NET but the releases targeting .NET Framework are also available
• IDE integration:
  • Visual Studio
  • VS Code
  • Sublime Text 3
  • Notepad++
• Debugging - the same experience as with any .NET app. Can be just triggered with css <script> //x. Or attaching the debugger.
• Extensibility
  • the default syntax C# and VB can be extended to support any other syntax with the alternative compiler ("Custom Compiler API")
  • support for precompiling to massage script before execution. Precompiler is just another script.
  • extending command line parameters: css -elevate script. -elevate is just another script
• Tunneling compilation to the stock C# compiles of your choice: csc, Roslyn, dotnet.
• Derivative tools
  • [CS-Syntaxer](https://community.chocolatey.org/packages/cs-syntaxer - service that is used for CS-Script integration with IDEs. Cand be used to allow your plugin users to edit scripts with Intellisense.
  • WixSharp - Framework for building MSI files with C# scripts
  • Windows Explorer shell extensions Shell-X

The only experience/feature that CS-Script deliberately does not do is REPL. Even though one of CS-Script user donated one for .NET Framework edition. The reason for this is that REPL is a very simple tool that is available from quite a few third-parties. They are very decent and it is close to impossible to build another one that beats the competitors.

I hope it gives you a perspective and helps you to make your choice.

Cheers

[benblo]

Mate, it's like asking Coca-Cola CEO which soft drink is better 😄

Eheh I know, but from the few interactions we've had in the past I felt like you are a straight shooter, so I figured why not get the info right from the horse's mouth 😁 !

seeing dotnet-script having almost all cs-script core features repeated

Looks like I got my answer then. dotnet-script is likely good enough, unless you need advanced features.

My greatest joy is that no one is debating if we need scripting.

Right, that's another point, IMO in the last few years MS has done a great job of catching up with the real world (.NET Core, GitHub, VsCode, etc), and has been surprisingly fast and agile for such a big unwieldly corporation.
So now they're taking scripting seriously, could they just blow past CS-script in a couple years?

I think what I'm trying to say with this post too, is that maybe your front page should mention all this, a "disclaimer/state of the art" section at the top: what CS-script solves, how it's different from its competitor (where it's better, where they're similar), and also maybe why CS-script exists (history that led to this odd situation).

I may try to take your bullet list below and compare for myself with dotnet-script (I'll probably pretend like scriptcs doesn't exist). Another big thing you didn't mention is performance too: I mostly write one-off scripts, so iteration is key to me, and in the past (at least 2-3 years ago) I've found first-time runs to be slower than ideal (that's one area where Python still has a leg up)... compilation is compilation right, but I'd be interested to see if dotnet-script can do better there.

---

Tunneling compilation to the stock C# compiles of your choice: csc, Roslyn, dotnet.

Wait, what? Roslyn and dotnet are 2 different compilers?

The only experience/feature that CS-Script deliberately does not do is REPL.

Fine by me, I never got into REPL, in any language... maybe I would with C# though, never really thought about it! In any case I'd probably just use dotnet-shell then.

[oleg-shilo]

Yep, it looks like we are in the agreement then.
Just to give you the answers to some of your question...

Tunneling compilation to the stock C# compiles of your choice: csc, roslyn, dotnet.

Wait, what? Roslyn and dotnet are 2 different compilers?

Three to be precise. And they give the user some unique scripting experiences.
BTW using csc.exe as an engine, allows CS-Script compilation of the scripts on the "built-in" build server and squashes the first time compilation to 300ms instead of 1800ms (without the server) or even ~8 seconds for dotnet.exe heaviest launcher.

Or you can avoid installing SDK and use just .NET runtime if you use roslyn.

The details are here: https://github.com/oleg-shilo/cs-script/wiki/Choosing-Compiler-Engine#compiling-engines-summary

[oleg-shilo]

BTW, I have updated the README.md with the disclaimer and the link to the features overview.

[benblo]

The details are here: https://github.com/oleg-shilo/cs-script/wiki/Choosing-Compiler-Engine#compiling-engines-summary

Hmm, so if I understand correcly:

• csc is the best for iteration
• Roslyn is a close second, and is good for deploying to end users without requiring the SDK
• dotnet is good for... nothing?

BTW, I have updated the README.md with the disclaimer and the link to the features overview.

You know... I hadn't even seen that overview page yet 😅, so thanks for that!

[oleg-shilo]

Correct, except for one thing - dotnet
It's good for two reasons:

• the only one that can compile XAML (MAUI, WPF)
• it supports any exotic feature that is available in .NET but has nothing to do with C# is automatically supported by this engine.

While first compile time of 8 seconds is shockingly bad (on a good machine 3.5 seconds). I can see how in some scenarios you may choose this option.

[damian-666]

I looked at this all day, have some stuff to share about my 90% complete but successful use case and scripting in general, and the contentious issue of hot reload, debuggers, IDEs that cost money, and coredebug...

re: My greatest joy is that no one is debating if we need scripting. again, thanks for keeping such a good humor and giving .net what they alwasy needed all these years. useful for experimental work like fluid dyanmics and AI with tensorflow in C , but with a python front end, but, leaving you wiht something you can deploy and ship, and now on about 10 platforms, with a visualizer like monogame, in a custom CAD like IDE with docking windows, and entity plugins, its great. now net7 is giving me generalized SIMD intrinsics , i have performance gens of 50% this year by going a big 6 week step to Net6, then effortlessly to Net7 last week by changing 6's to 7's.

on an aside:
sadly in the meantime when .net was windows only, Julia came out to meet those needs and scientists have embraced it, most don't know c# . net exists. so I used to ported their c and Java code to c# for fluid dynamics or such.. now they use julia, which might get .net support , or a JIT converter, or maybe never. Julia boasts the scripting of python + the performance of c++. but i doubt JUlia foundation can compete with .net 7, generalized SIMD intrinsices n Vector , gpgpu in Silk all that stuff being developed in Net foundation.

So... scripting in CS and .net lives on.. an because of politics between JetBrains and Microsoft.. i think it will need 3rd party support for a while.

  There might be a core debugger from Samsung which i will get into at the bottom of this sorrowfully TLDR comment..

In own use case if your are curious, problems almost all solved: updating this an the other stuff i had. i'll share if you care, otherwise skip to the debugger bit at the end...

for me i now have everything I really need with cs-script AvalonEdit updated, with the debug.Writelin, the Debug.Break,, injected in th escript and some UI cut from AvalonEdit. my remaining issue is codecompletion and getting off NRefactory and the old c# syntax.

so all my old AvalonEdit based stuff is working.. in wpf edit, since its slower now to activate a script after it does preparse, i added buttons to toggle off CSharpParser.SupressCodeAnalysis,, so if i dont want to debug or havent attached one, i shut it off... in my case CodeAnalyiss hasn't any benefit.. and make its slower . its fine.. if the code has errors from new c# sntax i can do a force build and wait, it maybe report errors or just work .. . i need to reference more assemblies more i think for better intellisense. I added a button to switch to CodeDom, that might give me a sort of way to HotReload into the main appdomain, not sure. probably never use it though. I solved the css type prefix i mentioned earlier simply by casting to object for the parameters. The class serializes the same regardless of script or precomilied, wiht a name i provide as attribute. Reflection should give me the properties so im fine, it doenst need strict type i can use reflectoin or type casts to either. so i can load parameters from script that were saved from the original assembly..
,
So remaining issues are : Codecompletiion and having to use an external debbugger.

havet tried the Syntax tree yet , but i want to look for a complete solution with moer intellisense and mabye even debugging first, and i so I firsr evaluate Notepad ++ and Rosyln pad code code completion, intellisense, and i think i should dig more into roysln pad. Avalonia edit... next.. because the light bulb , intellisense the wpf UI in royln pad is better adn similar to what i have in hy codebase and VS2022 now , and ill then use the CSScript dong the final build... i only care about c#. ill find a solution ...

but The real issue is debugging/ inspection... using two tools is a huge context switch.. inspection with writeline sucks but i could probably make a watcher UI rathr than a logger window .... anways i dug to the bottom and i got almost something:

1. tooltip, inspection in visual studio from temp assemblies doesnt work. (but vs 2022 watch dialog, does work) .. Hot reload isnt really an option it doesn't work.. mabye wiht CODEDOM it would.. sort of replace the code and then inspection woutl work better.. but regardless using a separate debugger UI is bad.. Everyone wants a a way to integrate a debugger. So i search and come up wiht almost something....

debugging c# outside of VSCode or VS studo or jetbrains who sponsor and have payrolled staff in NetFoundation and compete, its a contentious huge issue wiht legal complications.. but there is a way. its amazing it exists at all...adn so powerful because of new cloud businesses models i guess...

**NOPE:
roslynpad/roslynpad#42 (help wanted for debugger since 2016)

https://github.com/eclipse/aCute#concept : mabye, havent looked...

https://github.com/Samsung/netcored : the core piece.

So if anyone wants to dig deeper or has other vautls i should look at, this is where i stopped for now ..**

sorry for the typos im just really overworked now...

[oleg-shilo]

Good stuff. Thank you for sharing.

Keep in mind I created CS-Syntaxer so I never need to reverse engineer Roslyn-based intellisense ever again.

[damian-666]

um, just reporting more on the state of debugging scripts since i never found a solution, but seems a major structural shift is in progress at .net foundation, so im curious for an expert's opinion on this , now im sort of future proofing my own roadmap, triaging stuff..

Q: Do you think that a debugging script host should be using the [ LSP protocol ?]

https://github.com/julia-vscode/DebugAdapter.jl/blob/master/src/debugger_requests.jl

does it seen like an ideal that won't be realized soon for .net? require major, months long overhaul to existing syntaxers? julia IDEs have already moved to this i believe.
I'd love to find an example of one built around this , for c# especiallly a wpf control, that can insert a debug bfreakpoint and step in or over- without - attachng an external debugger. And to avoid using some other adhoc manner, like inserting Debug.Writelin, sleep , wait until cancel rudimentay breakpoint, and other limited inspection code via reflection.

and Might that mabye allow julia scripts, other langauges ( even unmanaged?) , and .net hosts to speak to each other? and for debuggers and custom tooling IDEs to be buld around that standard? it looks to me like it includes the hooks for code completion, and mabye other domain- specific stuff for a particular application. id love to be able to add julia scripts even if not as binary components. I don't care about VB, but mabye f#.. right now, im all c#.. But i wont proceed with my debugger plans in an adhoc way, if someone else will do it later, adhering to this standards, so that i can live wiitout it then update to .net 8, without my custom IDE limited or breaking.

There was an official announcement about MSFT Omnisharp mvoing to that LSP standard, for VSCODE , and it provoced a big flame thread.. since they said some intellisense features will be closed source, tons of peope were triggered , saying that .net and MSFT is tryng ot lock them in to a specific toolling for short term profits, but i think its just the opposite, I think thats reasonable and progressive, if not essential , for .net ecosystem, but based mostly on my intuition.. theres AI , telemetry, privacy, security concerns, involved, and it seems like quite a big, contentious subject, especially if misunderstood or miscommunicated.

i dindnt link the MSFT announcement and flame because it will link trace back here, but u'd find it on google and my take is going to be have many thumbs down for not jumping on the MSFT bash mob, as i tend to ramble, get labelled as disruptive even when im offering praise, but i make typos, and fail to be concise, but you seem to understand me anyways :)

im on .net 7 and CS-Script and everything basically works except preparsee new c# systaxx, while this pans out...as i don't know how much work is involved for me to to update the preparsing/intellisense..

• if you have time, incentive , to look into this, thx ..therers so much going on....

[oleg-shilo]

Ho Damian,

I have to say that the points you are describing are the very points I also have my focus on. :)

• Debugging experience without integration with an external debugger.
  Very attractive goal but very challenging.

  • Break and discover a current frame

    1. A break can be a real one. All threads are suspended. In this case, a debugger needs to be attached. The debugger does not have to be a comprehensive one like IDE. It can be some very small executable embedded in the script engine. Or another instance of the engine. Similar to CS-Script.Npp plugin. Discovering objects of the frame is difficult as there is no available sample/documentation and I do not want to go through this pain again.
       Alternatively one can sacrifice the convenience of the discovery and instead request the runtime object(s) via eval and then use the whole power of reflection to discover the concrete object (easy).
       It's a scenario that will need to be ported on all supported OS..
    2. An alternative is a soft break (loop-with-sleeps). Only the current thread is blocked. No need for any debugger. Frame discovery is not possible. Thus we can rely on the same approach: using eval to get runtime objects and then Reflection. It's a portable scenario that will work seamlessly on any OS.

    Integration with any UI/CLI (DBG-viewer) can be via any protocol including LSP.
    The only difficult part that needs to be POC-ed "eval to get runtime objects"
    Trigger for that can be easily implemented via true break-point for option-A or an injected conditional loop-with-sleeps routine (option-B). In both (or in option-B only) AvalonEdit-approach is an adequate injection method.

  • Proceed to next step
    Only possible if a true debugger is used. relatively easy if the problem of the debugger is solved.

• LSP standard for OmniSharp.
  This is where I am comfortable. Since I have invested in Syntaxer with minimalistic but very adequate intelligence support based directly on Roalyn code analysis API. Thus it does not matter what happens with OmniSharp. CS-Script/syntaxer is immune.

• ".net 7 and CS-Script"
  Very easy. Just to rebuild and release. I am only waiting for .NET 7 to be officially released.

[oleg-shilo]

I have done some experimenting and was able to solve one of the first problems - mapping StackFrame variables to the variable names. It was in fact non-trivial problem as it required reading portable-pdb.
The solution can be found here: https://github.com/oleg-shilo/Stack-Analyser

Thus if a poor-man debugger concept is to see the light then the following fundamental POCs need to be accomplished:

1. Discovery of a call stack frame
   a. List of all frame local variables
   done
   b. Access to the local variables (frame objects) for reading and updating.
   not done yet
   c. Further discovery of the frame objects
   done in cs-script.npp extension

2. Integration interface to allow building clients
   not done yet, syntaxer like interface can be considered

3. Basic CLI and UI (maybe MAUI or web) client.
   not done but some ideas can be taken from cs-script.npp

As you can see only 1.b is really problematic. I did not find the solution. At least yet.
Feel free to test your luck at the playground repository I have created for that.

[damian-666]

Thanks for your advice, I was off my rails with "future proof " worries.. its a really hard problem , both, debugging and about the backbone framework for code completion, debugging ,looking in netcore histories, this "Babel" abstraction they want ,especially wrt to LLVM is kind of unsolved, has things like Vector3 with posits and other numerical types net 7 doesn even have.. etc.. several Archived attempts since 2015 so probably aint gonna happen, soon or ever..

For simpler languages. if it takes an MxN ( features vs languages) problem to a M complexity problem, nice goal ,I've never been there but the brochure looks nice. so agreed.the Omnisharp masterplan doesn't matter like you said.

so I plan to do exactly as you advise, try Syntaxer ,improve code completion basics, and test more options to speed it up, and abstractions on CS-script and then 2. But i have an practical simple idea for the debugger...with an idea for. Step In, Out, and Over (below)

your suggestion 2 sleep loop Breakpoints, because it's actually better for my case.. in my use case, a frame is a slice in time.. just model state i really need to inspect , not locals or stack frames. .mayge the caller metadata stuff I saw earlier might give me extra stuff for a call stack display.

i just usually want to inspect my model and controllers state , the inspectors use the UI thread/ wpf... so ..i do i realize now i can step over / out / and in... at least in script.
In my use case that isnt very uncommon (ML, DataScience CAD, CFD, Level Editors for games, scientific visualization Simulation ) , as an special purpose IDE for small parameter or behavior tuning plugins or entity extensions , and a general code builder /ide for architecture of big systems are very different use cases. So i realize i dont want/ really need a real debugger or complex code completion or the mad Git Copilot AI thing (i've tried it ),and i dont want to block especially the UI /Dispatcher thread or my ui will freeze and along with my text editor.." Frame" in the simulation and visualization, case doesn't care much about local variabe state ..

STEPPING:
Like you said, inject some code to put a Sleep loop thing that doesn't block the dispatcher, but will not allow callers Update(dt) method to get past the fake BP and exit the method. This will freeze the physics simulation or the drawing updates, but not the UI if I'm careful on the UI thread- called draw update interface methods that are scripted , to let the dispatcher run the pump to the ui as it loops in in thr fake BP. Then I can type code as well into the script ( I think) without having to -really- stop any threads or "Game loop" Then i can toggle it off, and say if i wanted

to step to next, just inject another fake BP after the next statement via parsing for the next statements to the ";" then toggle off the first, and that's a step over.

to step In, first put one after the next line as before. But then **Search or parse for the method I'm going to enter which must be in my script but that's usually my focus in that session.. overloads can make this maybe difficult or just put them on all the methods, keep a list to remove after. and then put another fake BP after the first { , then release the top one. so now im Stepped in.

To step out.. just means toggle just the last BP placed .. it should revisit the one i put after the caller**. Then I remove all the entry bps i put in the methods. So it's all doable I think. Hope i will find time to try this and report if it worked.

Problem would be that when I adjust the script or toggle injected code like this , it recompiles and reloads the plugin, so in my case it will revert the state and called the Loaded method..
so the script/plugin - extended object jumped back in time. so i might need to add all these waypoints in the script from the start, and use a kind of internal switch as you mentioned earlier, or the pipe process thing, as i really want to stop,trace,continue a single "frame", without reloading the plugin..

Interesting Frame state for me is my model and controller data. its a slice of time usually i put a break in Update(dt) . I need to just stop the simulation and look at the physics objects state,mabye some neuron nodes, filters, feedback control biases, things like that and most those have properties most that notify on changed anyways.. I can visualize with custom visualizers in WPF, property grids, I already use INotifyProperty changed on most . Or an reflect and use mabye the CallerMeta Data will help.

So if i get to try this ,ill report and i might have a better experience than vs VStudio debugger that never hotreloads or edit and continue, anymore , and the git copilot AI is spitting crazy Git hub plagiarism bloat..90% of the time.

Also for GPGPU and compute shader and tne new Net 7 Generalized SIMD Intrinsics, and other parallel stuff, most practicioners say they don't use stepping debuggers. We need to script and visualize. LIke with Shadertoy.. For the GPGPU or compute shader code.. the SIMD and the C# code that loaded params to the shaders, scripting and seeing changes to those in a custom IDE . Hopefully after i finsih prototyping fluids in my case ill might see huge gains from this stuff.. low effort 2x on vector2 20x,-1000x even, if I learn to use a platform generalized compute shader.

As far as Ui and IDEs like dev studio, I need all that stuff the undead wpf has. The docking and access to the backing texture unlocked so I can use.my own graphics renderer, and drive simulation in background threads at 4000 fps while drawing the last unlocked frame at 60 or 120hz hz ,whatever the tv baby wants.Via producer./ Consumer pattern.
I I can't bet on Maui at all..Xamarin is all bloat in my case. It takes me 2minites to launch a debug session on Android and 2sec on Netcore .

So I'm using the undead wpf net6 open source ,but it's windows only. There's Avalonia too..I've asked netcore if they might make wpf target open gl , drop SharpDx, or if Silk Windowing abstractions can make some of this more likely.. Doubt it though. but now I'm starting to use touch and stylus more and more ..and wishing for low code techinques like those connectanle custom node graphs for pipelines flows and controller loops .. so I'm clearly interested in maybe doing my IDE on tablets or when I fold my.surface studio to tablet at.least for debugging and tuning, so if Silk Windowing keeps advancing I might be able to one day realize a multiwindow IDE that works on phones , tablets and all PCs too.

But for now the existing scripting Net7 might be enough . This i mentioned is way out of my realistic scope..

wish I could be more concise , and
thnaks for reading my tangentially relevant remarks , and for helping me focus and not worry about futures , I actually already have enough for a release 2. In fact I think need to just fix bugs , tune some scripts , and ship :)

[oleg-shilo]

Glad to see you in such good spirits. 😄

BTW, 1.b in your case can be solved quite easily since you need to access globals but not local variables. Thus you can simply define some static object that has references to your model state. Thus you don't even need to go through the pain of StackFrame but rely on raw Reflection.

Good luck.