Oh, I see. Still, I managed to solve the problem by creating a separate table like the way you said.
Here’s how I solved the problem.
-
Add “CLICKHOUSE_DATABASE” environment variable on the SNUBA side and receive all sentry schema and data created or migrated to a specific database
-
You can get the Create query for each table through tabix (I recommend this method) or you can work based on the metadata in the storage connected to the host.
-
When creating Replicated and Distributed tables, always add “on Cluster” so that all shards and replicas are also created at the same time.
-
After creating a separate DATABASE, create the Replicated~, Distributed tables. The default mergeTreeFamily has a corresponding ReplicatedFamily for each, so move them one by one. For example, in the case of ReplacingMergeTree, change it to ReplicatedReplacingMergeTree, etc., and in the case of a simple merge table, MATERIALIZED VIEW, it is kept as it is. (At this time, the combined table is based on the distributed table)
-
In the case of replicated tables, wrap once more with distributed tables for sharding. Each Distributed table points to a replicated table. In the case of me, the shardKey is the projectId
-
You can see that it is resharded by inserting the sentry data previously received in the distributed table after all the relevant tasks are finished. (At this time, the name of the distributed table must be the same as the original sentry table name)
-
Finally, change the “CLICKHOUSE_DATABASE” environment variable that was changed on the snuba side to the newly created database.
-
Make sure the data is well sharded and replicated
Of course, as you mentioned, when the version is upgraded, the table schema is changed. Maybe I have to do this every time to upgrade. However, in the production stage, it was determined that sharding was essential, so we proceeded with the work and confirmed that it works without problems. Thank you for answer!