Adding pagination

[data-envoy]

Hi, if I wanted to add pagination to this - do I need to reverse engineer existing solutions like

• algolina
• typesense adapter
• elastic search adapter

or, are there official docs? I can't find them.

[dekimir]

You'll have to reverse-engineer it. You can start with a working Algolia setup like this. When that works as expected, try to examine the requests it sends to the Algolia server as you paginate in your browser; it'll probably vary the page parameter.

Once you understand how it forms the request, you can then modify this code to extract the paging parameter and translate it into an SQL OFFSET clause.

This may sound scary complicated, but you can do it if you just go step by step. I can help answer any questions you may have -- just post them here.

[data-envoy]

ok, thanks for the tips. It sounds managable.

Looing at some src for the typesense adpter, I guess will also have to modify the response.

I'm considering using this in production (after some work).
I'll eventually have web and mobile app, this seems like the most efficient way to get Postgres search results.

Maybe a difficult question, but do you see any drawbacks to this, vs the alternatave?
I think the alt. is building my own ui and my own search api endpoint.

[dekimir]

I'm considering using this in production (after some work). I'll eventually have web and mobile app, this seems like the most efficient way to get Postgres search results.

Yes, that's exactly why I started this project.

Maybe a difficult question, but do you see any drawbacks to this, vs the alternatave? I think the alt. is building my own ui and my own search api endpoint.

Building your own will always be the most expensive alternative; better to reuse existing components that someone has already developed and tested. After looking at other existing alternatives (eg, Elastic's search UI), I've concluded that Algolia's InstantSearch is the best to work with.

The only drawback I can think of is in Postgres itself: it doesn't support fuzzy term search. This makes it hard to deal with misspellings, either in the queries or in the corpus. I think I know how I can add this to Postgres, but it's not a small project, and I'm delaying it until there's more interest.

[data-envoy]

I think you could use pg_trgm for fuzzy matches. I read about it at https://rachbelaid.com/postgres-full-text-search-is-good-enough/

I've also read about a multi-pass approach to building up the search results at https://postgresml.org/blog/postgres-full-text-search-is-awesome that you might find interesting - even if you think the ML aspect is overkill.

Lastly, but maybe off-topic a bit. You might be interested in how a typesense eg uses sentence transformers for semantic search https://github.com/typesense/typesense-instantsearch-semantic-search-demo . I suppose that concept could be ported over to postgres.

[dekimir]

I think you could use pg_trgm for fuzzy matches. I read about it at https://rachbelaid.com/postgres-full-text-search-is-good-enough/

I tried that approach, but I don't see how it works for multi-term or prefix queries. You'd have to write your own query interpreter that gets fuzzy results for each (unquoted) term, manually combines them, then sorts and paginates the results, all without leveraging Postgres indexes. And you have to manually update the known-word table.

[data-envoy]

Sounds like something for the back-burner. At least for not it's something that goes beyond my level of understanding.