Memtx transactions in depth

[Totktonada]

There is concern that the memtx transaction mechanism is more complex than one can think after reading our docs. So, the expectation of a person trying to understand the transaction mechanism will broken on the reefs of reality again and again.

This is the excerpt from the tarantool’s English chat in Telegram (2018-02-13), the talk starts from an another topic:

Alec Larson: Does box.once create a fiber implicitly?
Konstantin Osipov: no
Alec Larson: So I should wrap with fiber.create before doing network calls?
Alec Larson: I'm assuming network calls don't implicitly yield unless inside a fiber. Is that right?
Georgy Kirichenko: A network operation does an yield or return Eagain in case if the operation cannot be processed immediately
Georgy Kirichenko: At this time a fiber creation does an yield
Dmitry Sharonov: it is not very good idea to do something that may fail (like network call) in box.once, because box.once is not reiterable
Dmitry Sharonov: if your network call fails - box.once still won't be executed again
Alec Larson: So doing fiber.yield() immediately after fiber.create() is pointless, right?
Georgy Kirichenko: Just after a fiber creation this created fiber is on cpu (with caller yield), this is limitation of the current fiber engine implementation. After a callee fiber yields caller fiber gets execution back, but this behavior isn't strictly specified and may be changed in future
Georgy Kirichenko: And there is no point to do fiber yield just after start because caller fiber is already yielded with all side effects (memtx transaction cancelation)
Alec Larson: That info should be in the docs somewhere. 🙂
Alexander Turenko: There is tricky detail: the transaction can be not aborted in some cases even after fiber.create(). I’m don’t know tarantool enough to point specific cases, but if speaking non-strictly:

1. Before a first write in a memtx transaction.
2. In case of 'simple' fiber function that can be optimized out to being executed within the callee stack… or like so.

Alexander Turenko: Such cases (as well as peculiarities of working within tarantool’s console) makes understanding of tarantool’s transactions on experiments harder than it should be. We need to create the 'memtx transaction in depth' section or article I guess. @lenkis, what do you think?
Elena Shebunyaeva: I agree
Elena Shebunyaeva: Could you please put this thread into a doc ticket?
Alexander Turenko: Of course.

[Totktonada]

Since this discussion we have added fiber.new() function, which creates a fiber w/o scheduling it for execution (however everything that were said re fiber.create() are still actual). We also start to raise an exception is case of transaction abort with yield: tarantool/tarantool@131121c (before this a transaction was silently rolled back).

My initial point was about documentation re transactions in memtx: how they are connected with fiber yield, what is implicit and explicit yield (don't sure we should draw a line here in the docs), why I cannot commit a transaction when working in a console, why I sometimes can yield after box.begin() and the transaction will not rolled back (it is before first write). The documentation should answer those questions. Don't know whether it is so now (didn't look a long time to this part).

Other questions from the dialog are unrelevant, but I would file its own issues for them (if there are no any). We need to clarify how box.once() should be used and where pitfalls here.

We should give a general rule how to do a guess whether a function may yield (important note: the same function may yield or don't yield depending of certain conditions).

[Totktonada]

We looked at this again with @Onvember.

Q: How they [transactions] are connected with fiber yield?

Already answered:

The cooperative scheduler ensures that, in absence of yields, a multi-statement transaction is not preempted and hence is never aborted. Therefore, understanding yields is essential to writing abort-free code.

Q: What is implicit and explicit yield (don't sure we should draw a line here in the docs)?

Already answered:

There are two types of yields:

• implicit yields: <...>
• explicit yields: <...>

Q: Why I cannot commit a transaction when working in a console?

No answer, let's describe it.

Q: Why I sometimes can yield after box.begin() and the transaction will not rolled back (it is before first write)?

This is missed in the doc at all.

---

I still think that the article about transactions is hard to being understand by a person who is new in tarantool. However I don't sure how to and whether it is possible to make it better. I propose to also ask @Mons when he'll return from the vacation, maybe he has ideas.

At least things becomes a way better when a yield start to raise an explicit error when it is in memtx transaction (on commit): now it is possible to experiment around and get the exact rollback reason.

[Onvember]

It's time to translate these notes