Feedback request: Forge complex queries through custom entities

Forge Forge Announcements
,
22

Hi @ryan, we really appreciate the feedback, and will be looking closer to how we can improve this. When deciding on the direction for Storage API, we considered multiple options, and the decision was based on multiple factors including addressing concerns from wider set of developers, not just around API functionality, but also impact to compliance, scalability and security. With this EAP, we will be further assessing the value of this feature, and how we can improve moving forward. We also plan to offer additional DB solutions in future to provide more flexibility and options to our developers. It would be great to discuss in more detail with you to better understand your pain points and how we can help to address them.

Thanks,
Sushant

23

Hi @SushantBista, thanks for taking the time to respond. As a developer, I realize I can’t see the meticulous research and thought that goes into these types of decisions. I only want to give an outside perspective. I just hope I come across as respectful. I appreciate the work and passion you and the rest of the team are putting into the Forge platform.

I don’t have a specific use case right now but I am looking at migrating some of my old plugins to Forge. A lot of where I’m coming from is a general frustration with Forge. It feels like every corner of the Forge platform is filled with compromises that aren’t being addressed as the platform matures. It’s been a couple of years since Forge left beta and I haven’t seen a lot of progress. I’m sure you guys have put a lot of thought into this particular feature but it feels like this is another compromise being added to the pile for reasons I mentioned in my original post. Multi-tenant cloud-hosted storage is a problem that has been solved. I wish you guys would better leverage those existing solutions. It seems like it would be less work for you guys and it would do more to meet developer expectations.

1 Like
24

Hi @SushantBista,

do you think the end of March timeline is realistic? In my view it would make sense to also just release some of the features, e.g. adjusting the sort ordering (ascending, descending).

In my assumption, Forge storage uses DynamoDB under the hood? Then passing that sort option through is a single boolean.

As a side note: I find the plan to “offer additional DB solutions” a bit worry some. It feels that you are reinventing a lot of existing, polished and stable tooling. You might deliver interesting features, but they will take a very long time to release. I just got started with forge, but I am already feeling a couple of topics where it would just be easier for me to go with a Connect app.

Best,
Peter

25

Hi @ryan, please know that we really appreciate your feedback, and definitely acknowledge your frustration.

We will definitely take your feedback on board while we are assessing features and improvements to Forge storage. I agree on taking more learnings from similar services to better meet developer expectations.

26

Hi @PeterJuras, thanks for your feedback.

We are close to EAP release. A team member will provide an update on this tread soon (likely next week).

In regards to sorting - as part of upcoming custom entities release, we will support sorting (ascending, descending) for indexable attributes.

I just want to clarify my comment on “offer additional DB solutions", by which I meant that we understand the need from wider developer community on accessing DB solutions outside of key-value. We are currently exploring these in order to provide more options and flexibility to users, while balancing other security, performance and scalability requirements of a multi-tenant platform.

Thanks,
Sushant

27

Hi everyone,
I wanted to provide a quick update for Complex Queries through Custom Entities.

Firstly, we have locked in April 2023 as a target release for the EAP.

Secondly, there are a few changes; however the experience remains close to what we shared earlier. The changes are:

  • We are removing the partition method in favor of passing partition as an optional parameter to the index method
  • The Range method has been renamed Where
  • Sorting is no longer a parameter, but a function named sort()
  • We have added support for the data type Integer
  • The Float type is currently not supported for Indexing operations. We expect to solve this by the EAP release in April; if not, we will rectify this soon after with a release
  • Partition can now be multi keyed
  • filter() is now broken down for convenience:
    • orFilter()
    • andFilter()
  • The objects you can store as a custom entity can have a maximum depth of 31
  • The attributes you can use with custom entity can have a maximum size of 240KB
  • The format for defining attributes has changed slightly and now looks like this:
1entities:
2      - name: 'product'
3        attributes:
4          brand:
5            type: 'string'
6          price:
7            type: 'float'
8        indexes:
9          - 'price'                     # 'simple' index on attribute 'price'
10          - name: 'by-brand'            # 'named' index on attribute 'brand'
11            range:
12              - 'brand'
13          - name: 'by-brand-and-price'  # 'named' index with partition
14            range:
15              - 'price'
16            partition:
17              - 'brand'
18

We will be sharing more details and any additional changes as we move closer to the EAP release.
Please keep an eye on this space to get more details about how to participate.
We are really excited to get this in your hands soon and start receiving your valuable feedback.
Thank you!

1 Like