kmlabz/sentry-onpremise
kmlabz/sentry-onpremise
Archived
2
0
Fork 0
Code Pull Requests Releases Activity

feat: Instruct users to migrate TSDB (#430)

Browse Source 
Tested this in a Ubuntu VM. The output of `date` is not too pretty but at least localized (so D/M vs M/D is not confusing)

## What is the TSDB migration?

We're effectively deprecating all TSDB backends but `sentry.tsdb.redissnuba.RedisSnubaTSDB`. We cannot reasonably support any other backend due to the fact that we would have to reimplement each of the backends in Relay, which is written in a different language. Also, like with deprecating mysql support, we don't really have the capacity to support things we do not use ourselves.

## Migration

`install.sh` should rewrite your configuration automatically and define a cutover date such that no data is lost. Before the cutover date, data is written to two backends at once, Redis and Snuba, and read from one, Redis. After the cutover date, event-related metrics will be read from Snuba which matches what we have on sentry.io.

## Manual migration guide for TSDB

In case `install.sh` is unable to migrate your files you will be given basic instructions on the console that essentially tell you to completely delete all TSDB config and paste the new, standard one. If for some reason you cannot say goodbye to your existing TSDB config, please create a new issue in this repo and cc @untitaker on it.
This commit is contained in:
Markus Unterwaditzer 2020-05-12 12:02:40 +02:00 committed by GitHub
parent a74a0cb221
commit adda25ee23
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 40 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -76,5 +76,6 @@ data/
# custom Sentry config
sentry/sentry.conf.py
sentry/config.yml
sentry/*.bak
sentry/requirements.txt
relay/credentials.json

39
install.sh
View File

@ -107,6 +107,45 @@ if grep -xq "system.secret-key: '!!changeme!!'" $SENTRY_CONFIG_YML ; then
echo "Secret key written to $SENTRY_CONFIG_YML"
fi
replace_tsdb() {
if (
[ -f "$SENTRY_CONFIG_PY" ] &&
! grep -xq 'SENTRY_TSDB = "sentry.tsdb.redissnuba.RedisSnubaTSDB"' "$SENTRY_CONFIG_PY"
); then
tsdb_settings="SENTRY_TSDB = \"sentry.tsdb.redissnuba.RedisSnubaTSDB\"
# Automatic switchover 90 days after $(date). Can be removed afterwards.
SENTRY_TSDB_OPTIONS = {\"switchover_timestamp\": $(date +%s) + (90 * 24 * 3600)}"
if grep -q 'SENTRY_TSDB_OPTIONS = ' "$SENTRY_CONFIG_PY"; then
echo "Not attempting automatic TSDB migration due to presence of SENTRY_TSDB_OPTIONS"
else
echo "Attempting to automatically migrate to new TSDB"
# Escape newlines for sed
tsdb_settings="${tsdb_settings//$'\n'/\\n}"
cp "$SENTRY_CONFIG_PY" "$SENTRY_CONFIG_PY.bak"
sed -i -e "s/^SENTRY_TSDB = .*$/${tsdb_settings}/g" "$SENTRY_CONFIG_PY" || true
if grep -xq 'SENTRY_TSDB = "sentry.tsdb.redissnuba.RedisSnubaTSDB"' "$SENTRY_CONFIG_PY"; then
echo "Migrated TSDB to Snuba. Old configuration file backed up to $SENTRY_CONFIG_PY.bak"
return
fi
echo "Failed to automatically migrate TSDB. Reverting..."
mv "$SENTRY_CONFIG_PY.bak" "$SENTRY_CONFIG_PY"
echo "$SENTRY_CONFIG_PY restored from backup."
fi
echo "WARN: Your Sentry configuration uses a legacy data store for time-series data. Remove the options SENTRY_TSDB and SENTRY_TSDB_OPTIONS from $SENTRY_CONFIG_PY and add:"
echo ""
echo "$tsdb_settings"
echo ""
echo "For more information please refer to https://github.com/getsentry/onpremise/pull/430"
fi
}
replace_tsdb
echo ""
echo "Fetching and updating Docker images..."
echo ""