# Creating secondary index on data within a Map

**URL:** <https://forums.foundationdb.org/t/creating-secondary-index-on-data-within-a-map/1910>\
**Category:** Record Layer\
**Created:** [January 27, 2020, 1:16am UTC](https://forums.foundationdb.org/t/creating-secondary-index-on-data-within-a-map/1910 "2020-01-27T01:16:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![samitsawant](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@samitsawant](https://forums.foundationdb.org/u/samitsawant)\
**Post date:** [January 27, 2020, 1:16am UTC](https://forums.foundationdb.org/t/creating-secondary-index-on-data-within-a-map/1910/1 "2020-01-27T01:16:00Z")

</div>

Hi,

I am completely new to Foundation DB and Record layer. I was trying to understand secondary indexes through the existing documentation. I am curious to know how would be index a data which is unknown before hand.

For example… imagine a data model below…

```
message Outer {
	optional Inner static_events = 1;
         map<String, Inner> dynamic_events = 2;	
}

message Inner {
       optional Event known_a = 1;
       optional Event known_b = 2;
       optional Event known_c = 3;

}

message Event {
	optional string name = 1;
     ......
}

```

Creating secondary indexes for any of the properties (eg name) for the “Event” type nested within the static\_events is understood.  
Can we do the same for the the same type within the dynamic\_event map ?  
Can we write some sort of FunctionKeyExpression implementation to do that ?

Thanks,

Samit

---

<div class="post-metadata">

**Author:** ![alloc](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/alloc/32/9_2.png) [@alloc](https://forums.foundationdb.org/u/alloc)\
**Post date:** [January 27, 2020, 5:04pm UTC](https://forums.foundationdb.org/t/creating-secondary-index-on-data-within-a-map/1910/2 "2020-01-27T17:04:26Z")

</div>

In Protobuf, a map is actually implemented as a repeated nested message with two fields, one called “key” and one called “value”. The record layer allows you to then address them as such, and there are some constants and utilities included to make that somewhat easier.

I’m not exactly sure which indexes you’re interested in, but if you wanted to index just the “name” of all `known_a` dynamic events, you could do something like:

```java
Key.Expressions.field("dynamic_events", FanType.FanOut).nest(
   Key.Expressions.field(Key.Expressions.MAP_VALUE_FIELD).nest(
       Key.Expressions.field("known_a").nest(
           Key.Expressions.field("name")
       )
   )
)

```

And getting a little more advanced, if you wanted to have an index on, say, `(dynamic_event.key, known_b.name)`, you could do that with something like:

```java
Key.Expressions.field("dynamic_events", FanType.FanOut).nest(
   Key.Expressions.concat(
       Key.Expressions.field(Key.Expressions.MAP_KEY_FIELD),
       Key.Expressions.field(Key.Expressions.MAP_VALUE_FIELD).nest(
           Key.Expressions.field("known_b").nest(
               Key.Expressions.field("name")
           )
       )
   )
)

```

Note that the `known_b.name` is indexed here in a way that associates it with the `dynamic_event.key` that produced it specifically.

There are also utility methods for defining key expressions on either the keys, the values, the (key, value) pairs, or the (value, key) pairs of a map. (Those last two produce the same index entries except with the order flipped, which means that the former can answer queries of the form `key = x`, the latter can answer queries of the form `value = y`, and both can answer queries of the form `key = x AND value = y`.) For example, if you just wanted to index the keys of all dynamic events, you can do that with:

```java
Key.Expressions.mapKeys("dynamic_events")

```

Which is essentially syntactic sugar for:

```java
Key.Expressions.field("dyanmic_events", FanType.FanOut).nest(
    Key.Expressions.field(Key.Expressions.MAP_KEY_FIELD)
)

```

Unfortunately, none of those static utility methods play great with map types where the value is itself a nested message, so it’ll have to be done the long way if you need to nest into the value.

See: our javadoc on `Key.Expressions`, in particular all of the things with “map” in the name: [https://javadoc.io/static/org.foundationdb/fdb-record-layer-core/2.8.97.0/com/apple/foundationdb/record/metadata/Key.Expressions.html](https://javadoc.io/static/org.foundationdb/fdb-record-layer-core/2.8.97.0/com/apple/foundationdb/record/metadata/Key.Expressions.html)

---

<div class="post-metadata">

**Author:** ![samitsawant](https://avatars.discourse-cdn.com/v4/letter/s/13edae/32.png) [@samitsawant](https://forums.foundationdb.org/u/samitsawant)\
**Post date:** [January 27, 2020, 6:58pm UTC](https://forums.foundationdb.org/t/creating-secondary-index-on-data-within-a-map/1910/3 "2020-01-27T18:58:35Z")

</div>

> [@samitsawant](#):
>
> message Outer { optional Inner static\_events = 1; map\<String, Inner\> dynamic\_events = 2; } message Inner { optional Event known\_a = 1; optional Event known\_b = 2; optional Event known\_c = 3; } message Event { optional string name = 1; … }

Hi Alec,

Thanks for the detailed reply.

I made a mistake in my earlier data model. The dynamic\_events has “Event” as the type instead of “Inner”.

```
message Outer {
	optional Inner static_events = 1;
         map<String, **Event** > dynamic_events = 2;	
}

message Inner {
       optional Event known_a = 1;
       optional Event known_b = 2;
       optional Event known_c = 3;

}

message Event {
	optional string name = 1;
     ......
}

```

But from you reply it looks like it would still be possible won’t it ?

```
Key.Expressions.field("dynamic_events", FanType.FanOut).nest(
   Key.Expressions.field(Key.Expressions.MAP_VALUE_FIELD).nest(
       Key.Expressions.field("name")
       )
   )

```

---

<div class="post-metadata">

**Author:** ![alloc](https://sea1.discourse-cdn.com/foundationdb/user_avatar/forums.foundationdb.org/alloc/32/9_2.png) [@alloc](https://forums.foundationdb.org/u/alloc)\
**Post date:** [January 27, 2020, 9:21pm UTC](https://forums.foundationdb.org/t/creating-secondary-index-on-data-within-a-map/1910/4 "2020-01-27T21:21:25Z")

</div>

Yeah, exactly. That will produce an index on all names from all dynamic events, linking the name back to the `Outer` record from which it came.