Running Docker on the host network is not supported on GCB, which is breaking our builds over at getsentry/sentry. This patch removes the use of sentry-cli and does a manual `curl` request to create the test event.
`INTERNAL_IPS` is used to check whether to allow superuser access or not. Limiting this to the Docker internal network makes it impossible for anyone to reach admin pages with on-premise setup.
This is a follow up to #572 and it fixes#577.
This is a long-needed test that tests the whole pipeline from Nginx, Relay, to Kafka, and Snuba. The final missing piece is testing the Symbolicator integration.
This PR is also a follow up to #576 as it didn't solve the Relay issues fully (the earlier fix was a coincidence or is not as reliable as it seemed).
Fixes#486 (finally?).
Fixes#486.
I finally figured out what the issue was: a missing `post-buffering` option to let Relay finish it's POST request for auth. This PR supersedes #543.
This patch adds `INTERNAL_IPS` definition to `sentry.conf.py` by sniffing the network from eth0 and relies on this for trusted Relays instead of the ALLOWLISTED PKs. This removes the necessity of syncing Relay PKs to `sentry.conf.py`.
This PR needs getsentry/sentry#19798 to work.
I would like to be able to customize the configuration for my Sentry 10 symbolicator instance, which this change allows me to easily do.
See related: https://github.com/getsentry/symbolicator/issues/245
Co-authored-by: Burak Yigit Kaya <ben@byk.im>
Hi,
I've been through quite a few different ways of implementing this fix and settled on creating a variable to store the output of checking whether the zookeeper copy target folder exists and copying the snapshot file based on the copy target folder existing. I've ran quite a few manual tests for each option as well. Currently the PR sits on Option 3 from the below options.
**Option 1**
Judging from the [Jira issue](https://issues.apache.org/jira/browse/ZOOKEEPER-3056), it seems like the work around for zookeeper upgrades could be omitted entirely since the issue relates to upgrades from v3.4.10 to v3.5.4. I've tested removing the zookeeper workaround entirely and that install ran smoothly on a clean install of Sentry (no existing data) as well as an install of Sentry that currently has very minimal amount of log entries (roughly 100 log entries). Could we possibly remove the workaround entirely?
**Option 2**
The second option was to simply add a check to the currently [existing line](https://github.com/getsentry/onpremise/blob/master/install.sh#L178) of whether the copy target folder exists and perform the snapshot file copy only if the copy target folder exists. This is the least amount of code and possibly the simpler fix while also setting the `ZOOKEEPER_SNAPSHOT_TRUST_EMPTY` env var to `true`, however, some unnecessary calculations will be done to determine the `ZOOKEEPER_LOG_FILE_COUNT` and `ZOOKEEPER_SNAPSHOT_FILE_COUNT`.
**Option 3**
I've created a variable to store whether the copy target folder exists and proceed with the zookeeper upgrade workaround only if the copy target folder exists. This means that if the copy target folder does not exist, the env var `ZOOKEEPER_SNAPSHOT_TRUST_EMPTY` will not be set.
Fixes#547.
Co-authored-by: chamirb <chamirb@globalkinetic.com>
This patch brings back the HTTP/1.1 related settings for uWSGI to fix#486 as apparently Relay tries to talk to Sentry Web with keep alives where uWSGI terminates the connection unexpectedly. It also ports some configs for uWSGI and nginx from single-tenant.
Tested this in a Ubuntu VM. The output of `date` is not too pretty but at least localized (so D/M vs M/D is not confusing)
## What is the TSDB migration?
We're effectively deprecating all TSDB backends but `sentry.tsdb.redissnuba.RedisSnubaTSDB`. We cannot reasonably support any other backend due to the fact that we would have to reimplement each of the backends in Relay, which is written in a different language. Also, like with deprecating mysql support, we don't really have the capacity to support things we do not use ourselves.
## Migration
`install.sh` should rewrite your configuration automatically and define a cutover date such that no data is lost. Before the cutover date, data is written to two backends at once, Redis and Snuba, and read from one, Redis. After the cutover date, event-related metrics will be read from Snuba which matches what we have on sentry.io.
## Manual migration guide for TSDB
In case `install.sh` is unable to migrate your files you will be given basic instructions on the console that essentially tell you to completely delete all TSDB config and paste the new, standard one. If for some reason you cannot say goodbye to your existing TSDB config, please create a new issue in this repo and cc @untitaker on it.
This PR disables checking for the SSE4.2 flag, which may not be present if the server is running in a VM even if SSE4.2 is in fact available to use (ClickHouse/ClickHouse#20). The KVM check was obtained from a [comment](https://github.com/ClickHouse/ClickHouse/issues/20#issuecomment-226849297) in that issue.
If SSE4.2 is not actually available in the VM, then the installation script may fail in an unpredictable way. Perhaps we can add a keyword argument to the script to let the user decide if they want to skip the check.
This patch does two things:
1. Separate creating of Relay credentials from syncing them to Sentry config
2. Has a more flexible Relay credentials check and sync: look for the quoted public key in the config file, if it is there, assume this must be correctly set as it is very unlikely to have that random key in a different context with quotes around. The second one is to allow having other whitelisted relay keys by using an append method when adding the new key.
We regularly prune old Google Cloud Build images and also GCB registry is not accessible to everyone all the time (firewall settings, being in China, etc.)
Fixes#445.
