keys starting with 'mytable\' are sorted together (e.g. 'mytable\row1' , 'mytable\row2' , …)
Does this mean that a prefix such asmytablewill only be stored once?
For example, if I had 500 keys that all started with mytable, would mytable be stored for all 500 keys, or would it be stored for the first key beginning with mytable, and have subsequent keys just reference the first mytable?
No, the mytable\ prefix will be stored 500 times. The directory layer offers a way to reduce mytable\ down to a smaller prefix, at the cost of an extra read.
The feature you’re looking for is called “prefix compression”. It’s being actively worked on as part of a new storage engine called Redwood. Eventually, it will come to replace the ssd storage engine, so you’ll get prefix compression by default, but it will still take some time for the development to reach that point.
directory implementation is not part of the C API. It is implemented as an independent layer on each language bindings. AFAIK, there is no formal spec for the directory layer, so one would have to port the current implementation. But there is a tool called bindings tester to verify the implementation, more information can be found at Creating New Bindings thread