I am trying to get deleteRecordsWhere (and the async variant) to work but I am running into trouble. It seems that it works pretty different than query and isn’t maybe as well covered with tests? I am not criticizing just looking for advice.
In particular, trying to get delete to work well I had to effectively use recordType as a primary key prefix for all the nodes. This seems to be the best practice anyway, and makes the most sense (imo). However, once I did that, it wouldn’t allow me to delete anything that wasn’t an exact key match.
I stepped through the RecordWhereDeleter and it is throwing with the following conditions:
Here’s an example of a simple record type:
RecordType {'Snapshot', [RecordTypeKey, Field { 'persistenceId' None}, Field { 'sequenceNr' None}, Field { 'timestamp' None}]}
Here’s the QueryComponent (serialized) that I was trying to use
I guess I am wondering is it fine to forgo using this and generate queries, then iterate and delete specific records. There isn’t going to be a ton of deleting so I’d rather just move on from thinking about this, as the rest of the layer system seems to be pretty rock solid.
I ended up hitting this again, and was wondering if there is something else, for example, the ability to remove all records in a key range, like a deleting equivalent to scanRecords.
I get the impression that in general these facilities don’t exist because you end up needing something like the QuiCK queue processing system to run them safely within the various transaction limits.
It does not work in all cases to use a range delete to remove records in a key range, because of secondary indexes, which also need to be adjusted. For example, by removing the corresponding index entry in the case of regular value indexes. Something that scans the records and deletes them while also fixing each secondary index is going to be about as expensive as scanning records and calling delete record, as you are evidently now doing.
What deleteRecordsWhere aims to optimize is the case where not only are all the records to be deleted in a contiguous range, but the entries in each of the secondary indexes as well. Then you can do n+1 range deletes, one for the records and one for each index. The case it handles today is where the delete conditions are equalities on a left prefix of the keys for the records themselves and all of those indexes. This even works for higher-level aggregate indexes, provided they are grouped by the necessary keys, which is often the case when you want to do these efficient bulk deletes.
What it does not handle is other conditions that still maintain contiguity. Which, for an inequality like in your case, would mean that every index had that field at the end so that all the index entries were together as well. And that there were no higher-level indexes which don’t bucket them together. There is nothing impossible about this support; it just isn’t there.
An interesting special case of that constraint is where there are no secondary indexes at all on the target record type(s). Perhaps that is what you have going on. I don’t know that this is dramatically easier to implement, but I could imagine it being more common.
Just to say a big thanks for that response. You cleared up a lot of my personal confusions and now I am trying to work out if some subtle movement of items in the data model will enable those prefixes to be appropriately aligned.