Improve performance of the History page queries

[stuartc]

Coming off #968.

We have several places where performance of the History page query is hampered, here is a list of things I noticed:

1. We should not cast columns when searching, for example both log_lines and the dataclip tables have their columns cast into varchars - this is very expensive.
2. Finding the last run by joining on finished_at is exponentially less efficient depending on how many attempts & runs there are.

Proposed solutions

• We add tsvector/gin indexes on logs and dataclips in order to improve search performance. #1898
• Add an ordered DESC NULLS FIRST index on Runs finished_at to improve sort performance. #1899
• Add index to attempt_runs for attempt_id and/or run_id, currently the compound index doesn't appear to be used for these queries.

These solutions are distinct (and should be done regardless) from taking another approach which is creating aggregate tables

1. Create aggregate (or columns) that get updated when an attempt/run is created or updated. Removing the need to join and filter Runs by exit code and finished_at.

References:

invocation.ex:417 - list_work_orders_for_project_query/2

[jyeshe]

@stuartc, @taylordowns2000 back then when we were discussing about which partition to use for logs, I have talked about the benefits of easier records cleanup (drop expired partitions) and narrowing the query scope to a specific partition when user filters by date. For the incremental search on History Page approach, it might be an opportunity before the launch to switch the logs partitioning to by range.

[stuartc]

We'll need to make some synthetic benchmarks to know where our performance dropoff is unacceptable. Changing now will be quite a bit of work, and changing back (or any other flavour of partition) later will be even more difficult. We already have production data in the tables so we have to treat it as critical.