Watch semantics

I looked through the simulation tests and I can’t find anything that waits on a watch future outside of a retry loop. The documentation gives an example of waiting on a watch future outside of a retry loop. It seems that it’s possible that a watch future errors with future_version if the storage node gets partitioned from the cluster but not the client for example (see [this])(https://github.com/apple/foundationdb/blob/df0baa00662a9a048befb633d91722024ec05df3/fdbserver/storageserver.actor.cpp#L815)

What behavior can a user expect here? I think we should either

  1. Document the possible errors for a watch future
    or
  2. Transparently retry the watch within the client and have simulation tests that wait on watch futures outside of retry loops