Skip to content

Adding deterministic boundary #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from

Conversation

developingjames
Copy link

Hello, great project! I've been using it extensively with ChatGPT. One thing is that I'm working out of a github repo and shipping the files as mimeograms to ChatGPT. I don't want to constantly have to keep moving files over so I want to generate the mimeograms as files, store them in the repo, and then only copy.paste the mimeograms over when there are changes.

Unfortuantely with the UUID mimeogram markers the diffs are annoying. This adds an option to generate a hash for the marker so it's a deterministic marker and won't constantly change. Hope you like it.

I had some problems with the tests passing on windows, so I fixed them. I also am not a big python developer so I had no clue to use hatch to get things running, etc, so added some readme instructions.

All of this was done with copilot agent GPT-4.1. Tests pass and compiled executable does exactly what I was hoping for.

Happy to make any changes as you see fit, just let me know.

@emcd
Copy link
Owner

emcd commented Jun 28, 2025

Hi @developingjames . Thanks for the PR!

Interesting workflow... not one I had anticipated, but the hash markers make sense. I suppose that you're not using something like Gitingest because you need finer-grained file selection or round-trip support or you are working with repositories not on Github or which are private.

Sorry that the contribution guide is not more discoverable; it's in the documentation, but that could probably use a more visible link in the README. I'll work on sorting that at out in addition to reviewing the PR. Some relevant notes on Hatch installation and using the develop environment can be found here.

@developingjames
Copy link
Author

Wow, never saw gitingest before, that looks pretty cool. I'll try it out and see how it works with ChatGPT. The one great thing about the mimeograms is it gives gpt and easy way to give me back the modified files and I can update the repo with apply. I had to put more instructions in the project to avoid it from giving me partial files or do things like [section unchanged] and replacing 100 lines with that. When it's giving me a mimeogram response.

If you're interested I can share those instructions.

Copy link
Owner

@emcd emcd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some mostly minor comments and questions. Also, might be good to update the data/configuration/general.toml template and ensure that this option can be wired from configuration. If you use it all of the time, then you won't need to keep specifying it on the command line.

@emcd
Copy link
Owner

emcd commented Jun 28, 2025

The one great thing about the mimeograms is it gives gpt and easy way to give me back the modified files and I can update the repo with apply.

Cool. That's exactly why I made Mimeogram (for the round-trip support), because none of the other tools that I saw had this. (Afaik, Gitingest does not have round-trip support though it is quite advanced in other ways.)

I had to put more instructions in the project to avoid it from giving me partial files or do things like [section unchanged] and replacing 100 lines with that. When it's giving me a mimeogram response.

If you're interested I can share those instructions.

Please do. I've encountered that before as well and it is certainly annoying. Generally, though, if I can get the LLM to work in an artifact/canvas, then it will properly make partial updates to the right places without rewriting the entire mimeogram and eliding content. It is more tempted to elide sections of documents when it is responding with updates inline as part of the conversation flow.

@developingjames
Copy link
Author

developingjames commented Jun 28, 2025

The one great thing about the mimeograms is it gives gpt and easy way to give me back the modified files and I can update the repo with apply.

Cool. That's exactly why I made Mimeogram (for the round-trip support), because none of the other tools that I saw had this. (Afaik, Gitingest does not have round-trip support though it is quite advanced in other ways.)

I had to put more instructions in the project to avoid it from giving me partial files or do things like [section unchanged] and replacing 100 lines with that. When it's giving me a mimeogram response.
If you're interested I can share those instructions.

Please do. I've encountered that before as well and it is certainly annoying. Generally, though, if I can get the LLM to work in an artifact/canvas, then it will properly make partial updates to the right places without rewriting the entire mimeogram and eliding content. It is more tempted to elide sections of documents when it is responding with updates inline as part of the conversation flow.

- Always output the entire file contents for each `Content-Location`, line-for-line exactly as it would appear in your vault, including all unchanged sections. Do not summarize unchanged parts or replace them with placeholders like [unchanged] — the mimeogram must be the full authoritative file version. Do not provide partial replacements, excerpts, or patch-style diffs — even for small edits. This preserves file integrity, avoids version drift, and ensures each file can be dropped in as a single source of truth.
- When providing mimeograms, always output them as plain text blocks directly in the chat — never use tools or formats that render as canvas artifacts, editable panels, or code editors. The entire content must appear inline, as a single raw block, to be copied without formatting surprises.
- Do not use `canmore.create_textdoc` or `canmore.update_textdoc` for mimeogram delivery unless specifically requested. Use literal raw text in chat instead.
- When providing code examples inside a mimeogram, do not use raw triple backticks. Use indentation or other safe markers to avoid premature end of the mimeogram

These instructions helped get really accurate mimeogram responses with ChatGPT 4o

@@ -505,3 +534,75 @@ Other Projects by This Author 🌟
* `python-icecream-truck <https://github.com/emcd/python-icecream-truck>`_ (`icecream-truck <https://pypi.org/project/icecream-truck/>`_ on PyPI)

🍦 **Flavorful Debugging** - A Python library which enhances the powerful and well-known ``icecream`` package with flavored traces, configuration hierarchies, customized outputs, ready-made recipes, and more.

GPG Commit Signing for Contributors
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@emcd I've never done GPG key signing before, I added this section in case it helps others. If it's not helpful just let me know and I can ditch it.

Copy link
Owner

@emcd emcd Jun 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I am sure that this is probably helpful for a lot of people. I will look it over tomorrow. Might be able to point people at the Github docs for some or all of it.

@developingjames
Copy link
Author

Fixed the linter findings, I was getting an error from pyright sources, not sure what I I'm doing wrong. uninstalled/reinstalled it, removed the develop env and recreated it, no dice.

PS D:\Data\Development\GitHub\developingjames\python-mimeogram> hatch run develop:linters
cmd [1] | ruff check --quiet sources documentation tests
cmd [2] | pyright sources
C:\Users\pr0bity\AppData\Local\hatch\env\virtual\mimeogram\TTEShGjl\develop\lib\site-packages\pyright\dist\dist\vendor.js:2
exports.id=121,exports.ids=[121],exports.modules={2007:(A,e,t)=>{"use strict";Object.defineProperty(e,"__esModule",{value:!0}),e.AliasFS=void 0;const o=t(5086);class r extends o.ProxiedFS{constructor(A,{baseFs:e,pathUtils:t}){super(t),this.target=A,this.baseFs=e}getRealPath(){return this.target}getBaseFs(){return this.baseFs}mapFromBase(A){return A}mapToBase(A){return A}}e.AliasFS=r},7237:(A,e,t)=>{"use strict";Object.defineProperty(e,"__esModule",{value:!0}),e.CwdFS=void 0;const o=t(6825),r=t(5086),n=t(8169);class i extends r.ProxiedFS{constructor(A,{baseFs:e=new o.NodeFS}={}){super(n.ppath),this.target=this.pathUtils.normalize(A),this.baseFs=e}getRealPath(){return this.pathUtils.resolve(this.baseFs.getRealPath(),this.target)}resolve(A){return this.pathUtils.isAbsolute(A)?n.ppath.normalize(A):this.baseFs.resolve(n.ppath.join(this.target,A))}mapFromBase(A){return A}mapToBase(A){return

SyntaxError: Unexpected token *
at createScript (vm.js:74:10)
at Object.runInThisContext (vm.js:116:10)
at Module._compile (module.js:537:28)
at Object.Module._extensions..js (module.js:584:10)
at Module.load (module.js:507:32)
at tryModuleLoad (module.js:470:12)
at Function.Module._load (module.js:462:3)
at Module.require (module.js:517:17)
at require (internal/module.js:11:18)
at Object.o.f.require (C:\Users\pr0bity\AppData\Local\hatch\env\virtual\mimeogram\TTEShGjl\develop\lib\site-packages\pyright\dist\dist\pyright.js:1:2356)

@emcd
Copy link
Owner

emcd commented Jun 29, 2025

Fixed the linter findings, I was getting an error from pyright sources, not sure what I I'm doing wrong. uninstalled/reinstalled it, removed the develop env and recreated it, no dice.

PS D:\Data\Development\GitHub\developingjames\python-mimeogram> hatch run develop:linters cmd [1] | ruff check --quiet sources documentation tests cmd [2] | pyright sources C:\Users\pr0bity\AppData\Local\hatch\env\virtual\mimeogram\TTEShGjl\develop\lib\site-packages\pyright\dist\dist\vendor.js:2 exports.id=121,exports.ids=[121],exports.modules={2007:(A,e,t)=>{"use strict";Object.defineProperty(e,"__esModule",{value:!0}),e.AliasFS=void 0;const o=t(5086);class r extends o.ProxiedFS{constructor(A,{baseFs:e,pathUtils:t}){super(t),this.target=A,this.baseFs=e}getRealPath(){return this.target}getBaseFs(){return this.baseFs}mapFromBase(A){return A}mapToBase(A){return A}}e.AliasFS=r},7237:(A,e,t)=>{"use strict";Object.defineProperty(e,"__esModule",{value:!0}),e.CwdFS=void 0;const o=t(6825),r=t(5086),n=t(8169);class i extends r.ProxiedFS{constructor(A,{baseFs:e=new o.NodeFS}={}){super(n.ppath),this.target=this.pathUtils.normalize(A),this.baseFs=e}getRealPath(){return this.pathUtils.resolve(this.baseFs.getRealPath(),this.target)}resolve(A){return this.pathUtils.isAbsolute(A)?n.ppath.normalize(A):this.baseFs.resolve(n.ppath.join(this.target,A))}mapFromBase(A){return A}mapToBase(A){return

SyntaxError: Unexpected token * at createScript (vm.js:74:10) at Object.runInThisContext (vm.js:116:10) at Module._compile (module.js:537:28) at Object.Module._extensions..js (module.js:584:10) at Module.load (module.js:507:32) at tryModuleLoad (module.js:470:12) at Function.Module._load (module.js:462:3) at Module.require (module.js:517:17) at require (internal/module.js:11:18) at Object.o.f.require (C:\Users\pr0bity\AppData\Local\hatch\env\virtual\mimeogram\TTEShGjl\develop\lib\site-packages\pyright\dist\dist\pyright.js:1:2356)

Thanks. I am surprised that Pyright would have issues on Windows, given that it is Microsoft-developed. That is clearly a Pyright internal issue though. You could try to install Pyright outside of Hatch and then run pyright sources inside your clone of the project and see if that helps.

(I do not run Pyright directly on Windows; instead, I run it in Windows Subsystem for Linux VMs. So, it is possible that it has Windows-native issues of which I am not aware.)

@emcd
Copy link
Owner

emcd commented Jun 29, 2025

- Always output the entire file contents for each `Content-Location`, line-for-line exactly as it would appear in your vault, including all unchanged sections. Do not summarize unchanged parts or replace them with placeholders like [unchanged] — the mimeogram must be the full authoritative file version. Do not provide partial replacements, excerpts, or patch-style diffs — even for small edits. This preserves file integrity, avoids version drift, and ensures each file can be dropped in as a single source of truth.
- When providing mimeograms, always output them as plain text blocks directly in the chat — never use tools or formats that render as canvas artifacts, editable panels, or code editors. The entire content must appear inline, as a single raw block, to be copied without formatting surprises.
- Do not use `canmore.create_textdoc` or `canmore.update_textdoc` for mimeogram delivery unless specifically requested. Use literal raw text in chat instead.
- When providing code examples inside a mimeogram, do not use raw triple backticks. Use indentation or other safe markers to avoid premature end of the mimeogram

These instructions helped get really accurate mimeogram responses with ChatGPT 4o

Thanks! Probably the best path here is to make a library of prompts for various LLMs and extend the provide-prompt subcommand to let people choose which one to use based on target LLM.

Or, another option would be for people to provide custom prompts via the configuration file.

(Should be a separate PR, if you're interested in working on this. Otherwise, I'll add it as a backlog issue.)

@emcd
Copy link
Owner

emcd commented Jul 3, 2025

Thanks for the PR update @developingjames , I'll probably integrate your changes in the next couple of days. I've been doing some background work to improve visibility of the development documentation and improving support for Claude Code and Gemini CLI.

@developingjames
Copy link
Author

Sounds good! Let me know if anything else needs adjusted. I can't say my prompt instructions have completely fixed ChatGPT when it comes to mimeograms, still loving it but it /really/ wants to give me just what changed, haven't gotten that solved 100% yet.

emcd added a commit to emcd/python-project-common that referenced this pull request Jul 3, 2025
…URLs.

- Add standardized Contribution section to README template with links to
  issue tracker, pull requests, and development guidance
- Update contribution.rst.jinja to use published HTML documentation URLs
  instead of raw Sphinx sources (docs-1 branch)
- Point common documentation URLs to stable GitHub pages
- Add future enhancement note for docs-* tag support

Addresses several sub-issues from emcd/python-mimeogram#8

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
emcd added a commit to emcd/python-project-common that referenced this pull request Jul 4, 2025
- Add common validation.rst with comprehensive guide to hatch validation
  commands (linters, testers, packagers, docsgen, make-all) and workflow
- Update environment.rst to offer uv tool install alternatives to pipx
- Add Git commit signature requirements with GitHub setup guide and GitLab
  local verification documentation
- Update contribution templates to reference validation and commit signing
  requirements
- Restructure environment setup with Commands and Git Commit Signatures
  subsections for better organization

Addresses several sub-issues from emcd/python-mimeogram#8

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@developingjames
Copy link
Author

@emcd Hey, I started using Claude.ai and it has native integration with github which is awesome. Getting the files back out of it into my workspace was working fine with mimeograms but I kept hitting various token/usage limits because the full file was getting shunted back and forth.

Created this guy to produce a deltagram from the LLM back to my workspace. Obviously heavily inspired by your project: https://github.com/developingjames/deltagrams Let me know what you think!

Also, it's the first open source project I ever created so if you have any tips/suggestions/etc for me, lmk!

@emcd
Copy link
Owner

emcd commented Jul 4, 2025

@developingjames : Congratulations on your first open source project!

Overall, it looks good. Delta/diff formats were on my TODO list, but Mimeogram development largely fell by the wayside after Anthropic starting letting people use their Claude Pro subscriptions with Claude Code. Claude Code is my main AI development tool these days; I have hardly touched mimeograms in the past month. This is why my responses to you have been a bit lethargic. (I still intend to incorporate your patch, but I've been busy with some of my more foundational projects lately.)

As for tips:

  • You probably want to .gitignore your .claude/settings.local.json file; that is updated by Claude Code sessions and is not meant to be committed. (See this for example.)
  • You could let people download the binaries directly, if they are not too large. Unpacking archives is an extra step. To increase trust in the binaries, you could consider generating SHA sums on them and attesting the artifacts. (For example.)

Also, I noticed from your example deltagram that you appear to using AI for story worldbuilding. I've been doing the same, though in a LitRPG context rather than sci-fi one. Claude is actually a good creative writer (far better than any OpenAI model, including GPT 4.5, or xAI Grok 3, imo); people talk about it being a coding model, but it has other talents too.

@developingjames
Copy link
Author

Awesome, thanks! I'll incorporate your feedback. Yup, experimenting with it for a book, we'll see how it goes!

@developingjames
Copy link
Author

One question for you @emcd, if you use claude code do you find that it eats up your usage limit shipping tokens back and forth? I was hoping with the project github integration with claude.ai it would cache the files and not eat up so much of the token limit, combine that with the deltagram and that was the vision back and forth.

Claude code is great but I cap out all the time. Are you finding that at all?

@emcd
Copy link
Owner

emcd commented Jul 5, 2025

@developingjames : I definitely cap out with Claude Code against a Claude Pro subscription. (Have not yet convinced myself that a Claude Max subscription is worth it.) Usually I can make it about 2.5 to 3 hours through a 5-hour window before I hit my usage limit. Gemini CLI is not yet mature enough to serve as a trustworthy backup and I don't really care about Codex CLI, since the OpenAI models are not that good at instruction-following. That said, I use the "down time" to write code that I don't trust an AI to write well and I also prepare TODO lists and requirements documents to feed to the Claude for when the next 5-hour usage window starts.

For really simple tasks, I also have a Claude Github workflow, which counts against metered API usage rather than the subscription.

Sometimes I still feed mimeograms to xAI Grok 3 or OpenAI o3 and we talk in the web GUI. Or, I run Deep Research tasks in the web GUI to investigate various things for my next projects.

The Claude limits can be frustrating if you're trying to focus on something single-mindedly, but if you're willing to hop between tasks and AIs over the course of the day, they are bearable.

@emcd emcd closed this in b4f4402 Jul 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants