I think there are essentially two questions embedded in here: (1) how to search by keyword and (2) how to paginate. I’ll tackle those in order.
To search by keyword, the tricky part here appears to be the “prefix” part of the search, if I’m understanding correctly. You should be able to use the Tuple and Subspace layers to serialize the keys to the database in the correct order, and then it’s a matter of finessing the keys to read correctly. I think if your inserts are something like:
userSubspace := subspace.Sub("system", "user")
func (tr fdb.Transaction) WriteUser(username string, keyword string) {
key := userSubspace.Pack(tuple.Tuple{keyword, username})
tr.Set(key, []byte{})
}
Then reading is something like:
func (tr fdb.Transaction) ReadKeywords(keywordPrefix string) ([]string, err) {
keywordBytes := userSubspace.Pack(tuple.Tuple{keywordPrefix})
rangeStart := keywordBytes[:len(keywordBytes) - 1]
rangeEnd := fdb.Strinc(rangeStart)
fdbRange := fdb.KeyRange{Begin: rangeStart, End: rangeEnd}
// then read the range (as discussed below)
}
For the pagination part, I think you could do something like:
func (tr fdb.ReadTransaction) ReadPaginatedRange(fdbRange fdb.Range, continuation fdb.KeyConvertible, limit int) ([]fdb.KeyValue, fdb.KeyConvertible, error) {
options := fdb.RangeOptions(Limit: limit)
var effectiveRange fdb.Range
if continuation == nil {
effectiveRange = fdbRange
} else {
_, end := fdbRange.FDBRangeKeySelectors()
effectiveRange = fdb.SelectorRange{Begin: fdb.FirstGreaterThan(continuation), End: end}
}
rr := tr.getRange(effectiveRange, options)
kvs, err := rr.GetSliceWithError()
if err != nil {
return nil, nil, err
}
if len(kvs) < limit {
// Did not return all results, so no more data. Return nil as the continuation.
return kvs, nil, nil
} else {
// Hit limit. Return the last key in the slice as the last read key.
return kvs, kvs[len(kvs) - 1].Key, nil
}
}
My go is a little rusty, but I think this is essentially what you want. Modulo any compiliation errors (disclaimer–I haven’t tried to compile this), the idea here is that it will read some number of keys and then return a 3-tuple containing: the data, a “continuation” key, and an error. The error just propagates any error encountered from FDB; the continuation is then used to resume the get range, with a nil continuation passed into the function meaning “resume from the beginning” and a nil continuation coming out of the function meaning, “there is no more data”. So then you would call this with something like:
func (db fdb.Database) ReadPaginatedRange(range fdb.Range, limit int) ([]fdb.KeyValue, error) {
var continuation fdb.KeyConvertible = nil
var results []fdb.KeyValue = []
var done bool = false
for !done {
_, err := db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
kvs, continuationInner, eInner := tr.ReadPaginatedRange(range, limit, continuation)
if eInner != nil {
return nil, eInner
}
continuation = continuationInner
results = append(results, kvs...) // there's probably a better way to do this that doesn't have O(n^2) runtime
return nil, nil
})
done = continuation == nil
}
return results
}
(Again, sorry, my go is a little rusty.) But the idea is that a different transaction is used for each loop; retriable errors encountered while paginating (like transient network failures or an fdb recovery or a transaction hitting the 5 second limit) are retried by the ReadTransact call, but successful reads advance the continuation until the results slice has all of the results from all of the transactions. One still has to be at least a little careful to make sure that one doesn’t read too much data from FDB and OOM one’s process, but alas.
Something like that.