Inotify for windows

[gopherbot]

by xavier.mehaut:

Hello,
I saw with interest the sub-package os.inotify ; unfortunatly, it is only for linux up
to now... Do you intend to make a windows port of this library... If so, you can check
the jnotify (http://jnotify.sourceforge.net/) project which offers the same capabilities
for linux, windows and macos...
best regards
Xavier

[robpike]

Comment 1:

Owner changed to @alexbrainman.

Status changed to Accepted.

[alexbrainman]

Comment 2:

I have never used anything like that myself, and I don't know how it could work. But
quick search suggests ReadDirectoryChangesW could be a good fit here. This is very low
priority for me, but if you're interested to try and implement it yourself
http://golang.org/doc/contribute.html, everyone will help you along.
Alex

Labels changed: added os-windows.

Status changed to HelpWanted.

[gopherbot]

Comment 3 by xavier.mehaut:

i use personaly jnotify which provides also a wrapper for windows. All the source code
is available for calling the dll. Maybe it could help. 
I ll try to see next week if i ve time.

[alexbrainman]

Comment 4:

It uses ReadDirectoryChangesW alright. It also uses "completion ports" to monitor for
changes, but lucky us, we have goroutines, we could just block <g>.

[gopherbot]

Comment 5 by hectorchu:

My attempt at this can be found at http://golang.org/cl/4188047/.
If anybody can review and/or test it that would be great!

[gopherbot]

Comment 6 by xavier.mehaut:

Thanks a lot for this contribution ; I'll try to take time to test it soon ;

[gopherbot]

Comment 7 by crazy1be:

I've started on an API that should work with the subset of functionality provided by
both windows and linux, callback-based rather than channel-based. Currently only
supports linux, but making watchers for other operating systems should be trivial; they
need only satisfy the Watcher interface (see the README). Supports Modify, Delete, and
Create events currently. A Move event is planned, but not implemented for InotifyWatcher.
Let me know if you have any feedback or concerns about this API. Repository is at
https://github.com/crazy2be/fsmon.

[alexbrainman]

Comment 8:

I don't use this functionality myself, so it is hard for me to comment. But you're
welcome to start a discussion on golang-nuts, if you have some proposal in mind.
FYI, there is this change http://golang.org/cl/4188047/ that has not come to
anything yet.
Alex

[ancientlore]

Comment 9:

I am keen to try the changes in http://golang.org/cl/4188047/ if I can get my
Windows build working. What's the best way to move this toward release?

[alexbrainman]

Comment 10:

> I am keen to try the changes in http://golang.org/cl/4188047/ if I can get my
Windows build working. 
If you're asking how to build go for Windows, then you need mingw tools. You could
download them yourself: http://code.google.com/p/go-wiki/wiki/WindowsBuild, or you could
get pre-build zip file that Joe Poirier prepares
https://groups.google.com/d/topic/golang-nuts/u9h4TSs5mRk/discussion. Then you need to
fetch go repo with mercurial and build it.
Alternatively, I use linux/386 to develop and just cross-compile to windows/386.
> What's the best way to move this toward release?
As far as I remember, CL didn't go anywhere, because we couldn't decide on what the
interface should look like, so it is simple, reliable and works across of different OS
and OS-apis. Russ suggested we discuss that on golang-nuts. And that is where it
stopped. You could start discussion again if you have good suggestion on how to
implement it.
Alex

[gopherbot]

Comment 11 by crazy1be:

The difficulty here is that the interfaces provided by different operating systems
differ so much that making a single, coherent, interface is going to potentially result
in loosing a significant amount of functionality. In particular, we loose the advantages
each system provides, making implementing something that makes full use of the APIs
(e.g. a search indexing engine) quite difficult. However, that being said, we can easily
create an API that supports the most useful subset of functionality: watching arbitrary
directories for create, delete, and modify events.
The situation on Linux is probably the rosiest, which is part of the reason os/inotify
already exists. The inotify api provides events of almost every variety, for any
directory, in a reasonably coherent API. The go wrapper on this is pretty nice,
abstracting the low-level API surprisingly nicely.
On Windows, things are still pretty nice, if the function calls are a bit longer and
uglier :P. ReadDirectoryChangesW() appears to support the subset of functionality I
described above almost perfectly, and even has the additional feature of being able to
watch sub-directories natively, no additional directory-walking code required. I wasn't
able to figure out how you register to receive and then receive events, but It seems
possible that ReadDirectoryChangesW() returns all changes since the folder was opened or
it was last called.
On Mac OSX, the situation is UGLY. The API is completely and totally different from the
ones on Windows and Linux, and, it seems, designed for a completely different usage
case. It was implemented in order to allow the search indexer to keep up-to-date on file
changes, and was initially private to only that program. Since then, the FSEvents API
has been added, which allows external applications access to the same information that
the search indexer gets, filtered by top-level directory. This is recursive in nature,
and it is impossible to make it non-recursive (although you can obviously ignore events
from subfolders). Because you can ignore information, that is not a big issue. The
bigger issue is that this API does not provide you with any information on *how* a file
changed, telling you only that it changed. The jnotify library gets around this by
checking for changes itself, but this has obvious disadvantages in that you can only see
changes within the directory, and you're liable to miss events if there are multiple
events within a short period. Moves seem especially difficult to guess, i'm not sure how
jnotify gets around this.
All in all, I think that we should provide the low-level (syscall) and medium-level
(os/inotify) interfaces to each library, in case application developers want to take
advantage of this additional functionality on a particular platform. The wrapper library
should provide only basic functionality, like detecting modifications, moves, creations,
and deletions. I'm undecided whether we should support recursive watching, it certainly
has quite a few uses I can think of, and is supported directly on all OSes except Linux.
The interface for this library is tricky to decide on. I started on mine
(http://github.com/crazy2be/fsmon) with Callbacks, simply because that seemed like it
would work best for the way I was using the library. However, channels are also an
option, perhaps divided like this:
watcher := fsmon.NewWatcher(true)
watcher.AddWatch("foo")
str, str := <-watcher.Moves
err := <-watcher.Errors
str := <-watcher.Deletes
str := <-watcher.Creations
str := <-watcher.Modifications
Let me know of your ideas and thoughts on this, @michael.

[alexbrainman]

Comment 12:

This is all very interesting, but there are 7 people (who monitor this issue) that can
see your post. That's not enough. 
Please, post your suggestion on dolang-dev or golang-nuts. You should receive better
response there. I know nearly nothing about the technology. Might be able to help with
implementation. But we need to decide on api first.
Thank you.
Alex

[gopherbot]

Comment 13 by xavier.mehaut:

Hi
As, when I started an internal project, inotify for windows didn't existed,
I used instead JNotify which is portable and I compiled natively  the
project with excelsior jet. The result is efficient and simple to implement.
Maybe you could follow a same scheme...

[gopherbot]

Comment 14 by xavier.mehaut:

I personnaly think that recursivity is mandatory

[ancientlore]

Comment 15:

Thanks for all the feedback. We do need to move this to golang-nuts.
Alex, thanks for the links - I do have all that installed, just having some errors in
some tests. Hopefully will figure it out soon.
crazy2be, to receive events that a directory or subdirectory was modified in Windows you
use FindFirstChangeNotification/FindNextChangeNotification. It returns a handle that you
wait on using WaitForSingleObject or WaitForMultipleObjects. You still have to use
ReadDirectoryChanges to get the specific changes - these other functions just give your
thread a way to wake up when changes happen.
I personally like the initial inotify interface using one channel. Using a channel seems
more idiomatic for Go. Because you can filter the events, it seems odd to break them
into multiple channels - you'd have channels that never have messages. Plus, multiple
events could happen on the same file, and I am not 100% sure but I think the events can
be combined.
I'll start a topic on golang-nuts.