JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
a02424a220
storj/go.mod

62 lines
2.4 KiB
Modula-2
Raw Normal View History

Moved filepiece into storj (#66) * Moved filepiece into storj * Fix linter errors * Seek comment for linter * gofmt/golinter accidentally removed import * Fix small typos * Use the weird iota. P cool dude ✌️ * Do things the cool way * Changes requested by kaloyan * didn't need test main
2018-05-31 20:42:45 +01:00
module storj.io/storj
Light client implementation with get-info and list-buckets commands (#2) * Light client implementation with get-info and list-buckets commands * Fix client package name * Fix go.mod to work with vgo * Use single `fmt.Printf` for `get-info` output * Use unnamed import to client package * Simplify usage of sha256 and sha512 sums * Remove obsolete test code * Use helper structs for unmarshalling bridge info * Remove LGPL license files and adjust copyright headers * Use github.com/zeebo/errs * Use httptest for test http server * Use viper for env var management * Nested struct for swagger * Add github.com/zeebo/errs to go.mod * More bucket tests * word wrap long line * Use zeebo/errs for crypto errors
2018-04-16 14:42:06 +01:00
satellitedb: separate migration into subcommand (#3436) * separate sadb migration, add version check * update checkversion to do same validation as migration * changes per CR * add sa migration to storj-sim * add different debug port in storj-sim for migration * add wait for exit for storj-sim migration * update sa docker entrypoint to support migration * storj-sim satellite parts all wait for migration * upgrade golang-migrate/migrate to v4 because bug * fix go mod tidy
2019-11-02 20:09:07 +00:00
go 1.13
go.mod: enforce go 1.12 (#2399)
2019-07-01 12:57:05 +01:00
pkg/process: Now that we are trying to identify the root cause of the satellite load limitations (i.e. currently the satellite has a max ability of 400 rps for uploads and we need this to be higher), we are using the golang diagnostic tools to collect insight into what the bottlenecks are. We currently have a debug endpoint to gather some cpu and mem data, but it could be useful to have continuous profiling. GCP stackdriver has support for continuous profiling so lets set that up and see if it is helpful to gather more data. This PR adds support for [GCP continuous profiler](https://cloud.google.com/profiler) which allows enabling continuous cpu/mem profiling and the stats are sent to stackdriver in google cloud console. To enable the continuous profiling for a storj component, do the following: - prereq: the workload must be running in GKE and have Stackdriver Profiling IAM role permissions - provide the config flag `debug.profilename` in the config.yaml file for the workload (i.e. satellite api process, etc). The profilename should be the workload name, for example "satellite-api". - once the above config flag is provided, the profiler will be initialized and profiling stats will automatically be sent to GCP project where the workload is running and viewable in the Stackdriver Profile page in the console The current implementation assumes the workload is running in GKE, however if we find if useful we can add support to enable this from anywhere. But for simplicity, its configured this way assuming the main goal is to enable in production systems. Change-Id: Ibf8ebe2df7bf06fdd4951ee6a1e48854dd36ad47
2020-02-25 16:46:12 +00:00
replace google.golang.org/grpc => github.com/storj/grpc-go v1.27.2-0.20200225082019-bd19b647a81c
go.mod: use custom grpc that doesn't have tracing support (#2739)
2019-08-08 15:24:28 +01:00
Light client implementation with get-info and list-buckets commands (#2) * Light client implementation with get-info and list-buckets commands * Fix client package name * Fix go.mod to work with vgo * Use single `fmt.Printf` for `get-info` output * Use unnamed import to client package * Simplify usage of sha256 and sha512 sums * Remove obsolete test code * Use helper structs for unmarshalling bridge info * Remove LGPL license files and adjust copyright headers * Use github.com/zeebo/errs * Use httptest for test http server * Use viper for env var management * Nested struct for swagger * Add github.com/zeebo/errs to go.mod * More bucket tests * word wrap long line * Use zeebo/errs for crypto errors
2018-04-16 14:42:06 +01:00
require (
pkg/process: Now that we are trying to identify the root cause of the satellite load limitations (i.e. currently the satellite has a max ability of 400 rps for uploads and we need this to be higher), we are using the golang diagnostic tools to collect insight into what the bottlenecks are. We currently have a debug endpoint to gather some cpu and mem data, but it could be useful to have continuous profiling. GCP stackdriver has support for continuous profiling so lets set that up and see if it is helpful to gather more data. This PR adds support for [GCP continuous profiler](https://cloud.google.com/profiler) which allows enabling continuous cpu/mem profiling and the stats are sent to stackdriver in google cloud console. To enable the continuous profiling for a storj component, do the following: - prereq: the workload must be running in GKE and have Stackdriver Profiling IAM role permissions - provide the config flag `debug.profilename` in the config.yaml file for the workload (i.e. satellite api process, etc). The profilename should be the workload name, for example "satellite-api". - once the above config flag is provided, the profiler will be initialized and profiling stats will automatically be sent to GCP project where the workload is running and viewable in the Stackdriver Profile page in the console The current implementation assumes the workload is running in GKE, however if we find if useful we can add support to enable this from anywhere. But for simplicity, its configured this way assuming the main goal is to enable in production systems. Change-Id: Ibf8ebe2df7bf06fdd4951ee6a1e48854dd36ad47
2020-02-25 16:46:12 +00:00
cloud.google.com/go v0.52.0
cmd/statreceiver: Change v2 known metrics to toml config file Change-Id: Ib5e303210b2c0a59fad23ab4fd6333ea1aee9364
2020-02-18 23:31:25 +00:00
github.com/BurntSushi/toml v0.3.1
cmd/statreceiver: lua-scriptable stat receiver (#636) * cmd/statreceiver: lua-scriptable stat receiver Change-Id: I3ce0fe3f1ef4b1f4f27eed90bac0e91cfecf22d7 * some updates Change-Id: I7c3485adcda1278fce01ae077b4761b3ddb9fb7a * more comments Change-Id: I0bb22993cd934c3d40fc1da80d07e49e686b80dd * linter fixes Change-Id: Ied014304ecb9aadcf00a6b66ad28f856a428d150 * catch errors Change-Id: I6e1920f1fd941e66199b30bc427285c19769fc70 * review feedback Change-Id: I9d4051851eab18970c5f5ddcf4ff265508e541d3 * errorgroup improvements Change-Id: I4699dda3022f0485fbb50c9dafe692d3921734ff * too tricky the previous thing was better for memory with lots of errors at a time but https://play.golang.org/p/RweTMRjoSCt is too much of a foot gun Change-Id: I23f0b3d77dd4288fcc20b3756a7110359576bf44
2018-12-11 18:24:31 +00:00
github.com/Shopify/go-lua v0.0.0-20181106184032-48449c60c0a9
all: build release fully dprc and test in mixed mode Change-Id: I3bded3edf25a0b113601c8b12ecf1337f596649b
2019-11-11 18:58:08 +00:00
github.com/alessio/shellescape v0.0.0-20190409004728-b115ca0f9053
mod: upgrade miniredis to latest miniredis 2.5.0 had a bug with matching keys with newlines. Change-Id: I9bcf998459be6d7d4e03bca3589e989e5ed2304d
2020-02-06 12:56:23 +00:00
github.com/alicebob/miniredis/v2 v2.11.1
autoupdater: use blang/semver (#3063)
2019-09-20 15:22:40 +01:00
github.com/blang/semver v3.5.1+incompatible
Kademlia Package (#58) * wip * wip * get nodes * ping and moar * added GetNodes * added nodes to Kbucket * default transport changed to TCP * GetBuckets interface changed * filling in more routing * timestamp methods * removed store * testing * cleanup * go mod & test * deps * PR comments addressed * fix GetBucket * using SplitHostPort * Use JoinHostPort
2018-06-05 12:48:19 +01:00
github.com/boltdb/bolt v1.3.1
cmd/gateway: use a separate repository Change-Id: Idbb0b2b6cf0e60c6d5d91218c24524d72285cf26
2020-02-21 16:28:20 +00:00
github.com/btcsuite/btcutil v0.0.0-20180706230648-ab6388e0c60a
cmd/uplink: fix progressbar data race (#2987)
2019-09-10 09:51:49 +01:00
github.com/cheggaaa/pb/v3 v3.0.1
storagenode dashboard is working and passing tests locally (#1072) * storagenode dashboard is working and passing tests locally * linter fixes * linter fixes * moar linter fixes god * review fixes and updates * linter fixes
2019-01-16 21:09:13 +00:00
github.com/fatih/color v1.7.0
Fix go.mod (#395)
2018-09-28 16:44:02 +01:00
github.com/go-redis/redis v6.14.1+incompatible
update golang.org/x packages and grpc (#1363)
2019-02-26 15:49:56 +00:00
github.com/gogo/protobuf v1.2.1
satellitedb: separate migration into subcommand (#3436) * separate sadb migration, add version check * update checkversion to do same validation as migration * changes per CR * add sa migration to storj-sim * add different debug port in storj-sim for migration * add wait for exit for storj-sim migration * update sa docker entrypoint to support migration * storj-sim satellite parts all wait for migration * upgrade golang-migrate/migrate to v4 because bug * fix go mod tidy
2019-11-02 20:09:07 +00:00
github.com/golang-migrate/migrate/v4 v4.7.0
pkg/auth: use grpc.WithPerRPCCredentials (#2670)
2019-07-31 12:57:13 +01:00
github.com/golang/protobuf v1.3.2
Preliminary support for windows (#266) * Don't use url.Parse for bolt paths: filepaths may not be valid URL-s. * go.mod: update dependencies * README.md: add Windows instructions * pkg/overlay: check for the correct path and text in error * pkg/overlay: fix tests for windows * pkg/piecestore: make windows tests pass * pkg/telemetry: skip test, as it doesn't shutdown nicely * storage/redis: ensure that redis is clean before running tests
2018-08-23 08:05:56 +01:00
github.com/gomodule/redigo v2.0.0+incompatible // indirect
pkg/process: Now that we are trying to identify the root cause of the satellite load limitations (i.e. currently the satellite has a max ability of 400 rps for uploads and we need this to be higher), we are using the golang diagnostic tools to collect insight into what the bottlenecks are. We currently have a debug endpoint to gather some cpu and mem data, but it could be useful to have continuous profiling. GCP stackdriver has support for continuous profiling so lets set that up and see if it is helpful to gather more data. This PR adds support for [GCP continuous profiler](https://cloud.google.com/profiler) which allows enabling continuous cpu/mem profiling and the stats are sent to stackdriver in google cloud console. To enable the continuous profiling for a storj component, do the following: - prereq: the workload must be running in GKE and have Stackdriver Profiling IAM role permissions - provide the config flag `debug.profilename` in the config.yaml file for the workload (i.e. satellite api process, etc). The profilename should be the workload name, for example "satellite-api". - once the above config flag is provided, the profiler will be initialized and profiling stats will automatically be sent to GCP project where the workload is running and viewable in the Stackdriver Profile page in the console The current implementation assumes the workload is running in GKE, however if we find if useful we can add support to enable this from anywhere. But for simplicity, its configured this way assuming the main goal is to enable in production systems. Change-Id: Ibf8ebe2df7bf06fdd4951ee6a1e48854dd36ad47
2020-02-25 16:46:12 +00:00
github.com/google/go-cmp v0.4.0
satellitedb: separate migration into subcommand (#3436) * separate sadb migration, add version check * update checkversion to do same validation as migration * changes per CR * add sa migration to storj-sim * add different debug port in storj-sim for migration * add wait for exit for storj-sim migration * update sa docker entrypoint to support migration * storj-sim satellite parts all wait for migration * upgrade golang-migrate/migrate to v4 because bug * fix go mod tidy
2019-11-02 20:09:07 +00:00
github.com/gorilla/mux v1.7.1
satellite/marketing: Create New Offer (#2186) * update UI to reflect final mockups * implement create handler and render offers table data to UI * fix line-height unit and remove important from selectors * update file names and ids for clarity * shorten 'label' in ids * localize global vars, fix endpoint names, remove unnecessary receiver, fix comments * fix unnecessary initialization of pointer * correct file-naming conventions * register timeConverter in an init func for safety and remove unnecessary important from css * consolidate create endpoints and add comments * register timeConverter in init func * add copyright to files * introduce require pkg * add proper http server unit test * update linting and create offers concurrently in unit test * fix getOffers comment * add copy-right to unit-test * fix data-races * fix linting * remove converter in NewServer * requested changes in progress * add require for checking status code * renamed template file * fix 400 handler * fix missing copyright and remove extra line * fix build * run goroutine for testing parallel * evaluate reqType with switch stmt and promp for credit amount in cents * fix lint issue * add default case * remove unnecessary var * fix range scope error * remove empty lines and use long form for struct field * fix merge conflicts * fix template reference * fix modal id * not delete package * add currency formatting and requested changes * markup formatting * lean out currency logic and move wait outside loop * pass ToDollars func to home template * fix lint
2019-06-28 15:34:10 +01:00
github.com/gorilla/schema v1.1.0
go.mod: unlock graphql dependency and bump to latest Change-Id: I40026f6c8de155e024f5fbb51105546065393034
2020-02-25 11:17:49 +00:00
github.com/graphql-go/graphql v0.7.9
private/dbutil: retry single statements on cockroachdb This ought to make it so that all single statements (Exec- or Query-) on a CockroachDB backend will get retried as necessary. As there is no need for savepoints to be allocated or released in this case, there is no round-trip overhead except when statements actually do need to be retried. Change-Id: Ibd7f1725ff727477c456cb309120d080f3cd7099
2020-01-10 21:34:54 +00:00
github.com/jackc/pgx v3.2.0+incompatible
cmd/statreceiver: lua-scriptable stat receiver (#636) * cmd/statreceiver: lua-scriptable stat receiver Change-Id: I3ce0fe3f1ef4b1f4f27eed90bac0e91cfecf22d7 * some updates Change-Id: I7c3485adcda1278fce01ae077b4761b3ddb9fb7a * more comments Change-Id: I0bb22993cd934c3d40fc1da80d07e49e686b80dd * linter fixes Change-Id: Ied014304ecb9aadcf00a6b66ad28f856a428d150 * catch errors Change-Id: I6e1920f1fd941e66199b30bc427285c19769fc70 * review feedback Change-Id: I9d4051851eab18970c5f5ddcf4ff265508e541d3 * errorgroup improvements Change-Id: I4699dda3022f0485fbb50c9dafe692d3921734ff * too tricky the previous thing was better for memory with lots of errors at a time but https://play.golang.org/p/RweTMRjoSCt is too much of a foot gun Change-Id: I23f0b3d77dd4288fcc20b3756a7110359576bf44
2018-12-11 18:24:31 +00:00
github.com/jtolds/go-luar v0.0.0-20170419063437-0786921db8c0
all: use monkit v3 this commit updates our monkit dependency to the v3 version where it outputs in an influx style. this makes discovery much easier as many tools are built to look at it this way. graphite and rothko will suffer some due to no longer being a tree based on dots. hopefully time will exist to update rothko to index based on the new metric format. it adds an influx output for the statreceiver so that we can write to influxdb v1 or v2 directly. Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2019-11-08 20:40:39 +00:00
github.com/jtolds/monkit-hw/v2 v2.0.0-20191108235325-141a0da276b3
traces: fix memory leak for long running traces that aren't being collected for real this time. i'm so ashamed Change-Id: Ib05bb50d8e947dec2d872fd53e71eec561c2d0e8
2020-02-13 10:16:31 +00:00
github.com/jtolds/tracetagger/v2 v2.0.0-rc3
mod: bump common and some other packages Change-Id: Icf73220f5182f5a2fd1431471825948c300c0bea
2020-01-07 16:09:54 +00:00
github.com/lib/pq v1.3.0
Upgrade hrtime package (#869)
2018-12-17 11:41:40 +00:00
github.com/loov/hrtime v0.0.0-20181214195526-37a208e8344e
command-line benchmark and visualization (#342)
2018-09-11 16:35:46 +01:00
github.com/loov/plot v0.0.0-20180510142208-e59891ae1271
cmd/uplink: fix progressbar data race (#2987)
2019-09-10 09:51:49 +01:00
github.com/mattn/go-isatty v0.0.9 // indirect
mod: update sqlite module to v2.0.2 This updates SQLite amalgamation from 3.29.0 to 3.30.1. The module contains fixes for races. Change-Id: Ic6a06a43ba404de0091d8a2f7444a8f4b1d5d54c
2020-01-08 15:23:04 +00:00
github.com/mattn/go-sqlite3 v2.0.2+incompatible
CSR Service (part 2): cert signing rpc (#950) * CSR Service: + implement certificate sign rpc method + implement certificate signer client/server + refactor `AuthorizationDB#Create` + refactor `NewTestIdentity` + add `AuthorizationDB#Claim` + add `Token#Equal` + fix `Authorizations#Marshal` when marshaling identities and certificates + tweak `Authorization#String` format + cert debugging improvements (jsondiff) + receive context arg in `NewTestIdentity` + misc. fixes
2019-01-02 17:39:17 +00:00
github.com/nsf/jsondiff v0.0.0-20160203110537-7de28ed2b6e3
better batch-generation (#1219)
2019-02-06 08:04:12 +00:00
github.com/nsf/termbox-go v0.0.0-20190121233118-02980233997d
satellite/payments: allow floating point numbers for pricing Change-Id: I78b60134cf043746efef5371b761939a10f75aaf
2020-01-28 23:36:54 +00:00
github.com/shopspring/decimal v0.0.0-20200105231215-408a2507e114
V3-664 Extend User repository. Reimplemented with dbx (#591) * V3-664 Extend User repository. Reimplemented with dbx * structure updated * redundant packages removed, structure simplified * fixing goimports * removed constuctor from user struct * separated types declarations from database file * test file renamed * fixes according to review * fixing goimports * fixing goimports
2018-11-08 14:19:42 +00:00
github.com/skyrings/skyring-common v0.0.0-20160929130248-d1c0bb1cbd5e
all: use monkit v3 this commit updates our monkit dependency to the v3 version where it outputs in an influx style. this makes discovery much easier as many tools are built to look at it this way. graphite and rothko will suffer some due to no longer being a tree based on dots. hopefully time will exist to update rothko to index based on the new metric format. it adds an influx output for the statreceiver so that we can write to influxdb v1 or v2 directly. Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2019-11-08 20:40:39 +00:00
github.com/spacemonkeygo/monkit/v3 v3.0.1
cmd/uplink: use scopes to open (#2501) What: Change cmd/uplink to use scopes It moves the fields that will be subsumed by scopes into an explicit legacy section and hides their configuration flags. Why: So that it can read scopes in from files and stuff
2019-08-05 18:01:20 +01:00
github.com/spf13/cast v1.3.0
storagenode/cmd: create start graceful exit CLI (#3202)
2019-10-11 14:58:12 +01:00
github.com/spf13/cobra v0.0.5
Pointer lookup (#407) * Loads cache from context for PointerDB access * WIP adds overlay lookups to pointerdb requests * Pointer lookup code is added for Get * adds feature flag for pointerdb return * refactors pointerdb code * removes some unnecessary debug logs * Fixes indent in config * adds early return for non-remote pointers * formats code, removes some comments * Fixes tests broken by pointer proto changes * adds error check and merges variable declaration * removes commented out proto import * adds error check to pdbclient
2018-10-04 22:00:19 +01:00
github.com/spf13/pflag v1.0.3
cmd/uplink: use scopes to open (#2501) What: Change cmd/uplink to use scopes It moves the fields that will be subsumed by scopes into an explicit legacy section and hides their configuration flags. Why: So that it can read scopes in from files and stuff
2019-08-05 18:01:20 +01:00
github.com/spf13/viper v1.4.0
all: use monkit v3 this commit updates our monkit dependency to the v3 version where it outputs in an influx style. this makes discovery much easier as many tools are built to look at it this way. graphite and rothko will suffer some due to no longer being a tree based on dots. hopefully time will exist to update rothko to index based on the new metric format. it adds an influx output for the statreceiver so that we can write to influxdb v1 or v2 directly. Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2019-11-08 20:40:39 +00:00
github.com/stretchr/testify v1.4.0
satellite/payments - payment account setup (#3187)
2019-10-10 18:12:23 +01:00
github.com/stripe/stripe-go v63.1.1+incompatible
Improve logic for cutting the long tail during upload (#909)
2019-02-05 10:54:25 +00:00
github.com/vivint/infectious v0.0.0-20190108171102-2455b059135b
storagenode: monitor available space and bandwidth Change-Id: I5763597327c5b32982faab8910c136c6c8dc18c5
2020-02-09 00:04:23 +00:00
github.com/zeebo/admission/v2 v2.0.0
use-drpc: use protoc-gen-drpc to generate protobufs Change-Id: I5c23256068e30864022dba5137c499796ab9d6ad
2019-09-05 00:05:51 +01:00
github.com/zeebo/errs v1.2.2
lib/uplink: encryption context (#2349) * lib/uplink: encryption context Change-Id: I5c23dca3286a46b713b30c4997e9ae6e630b2280 * lib/uplink: bucket operation examples Change-Id: Ia0f6e69f365dcff0cf11c731f51b30842bce053b * lib/uplink: encryption key sharing test cases Change-Id: I3a172d565f33f4e591402cdcb9460664a7cc7fbe * fix encrypted path prefix restriction issue Change-Id: I8f3921f9d52aaf4b84039de608b8cbbc88769554 * implement panics in libuplink encryption code todo on cipher suite selection as well as an api concern Change-Id: Ifa39eb3cc4b3443f7d96f9304df9b2ac4ec4085d * implement GetProjectInfo api call to get salt Change-Id: Ic5f6b3be9ea35df48c1aa214ab5d355fb328e2cf * some fixes and accessors for encryption store Change-Id: I3bb61f6712a037900e2a96e72ad4029ec1d3f718 * general fixes to builds/tests/etc Change-Id: I9930fa96acb3b221d9a001f8e274af5729cc8a47 * java bindings changes Change-Id: Ia2bd4c9c69739c8d3154d79616cff1f36fb403b6 * get libuplink examples passing Change-Id: I828f09a144160e0a5dd932324f78491ae2ec8a07 * fix proto.lock file Change-Id: I2fbbf4d0976a7d0473c2645e6dcb21aaa3be7651 * fix proto.lock again Change-Id: I92702cf49e1a340eef6379c2be4f7c4a268112a9 * fix golint issues Change-Id: I631ff9f43307a58e3b25a58cbb4a4cc2495f5eb6 * more linting fixes Change-Id: I51f8f30b367b5bca14c94b15417b9a4c9e7aa0ce * bug fixed by structs bump Change-Id: Ibb03c691fce7606c35c08721b3ef0781ab48a38a * retrigger Change-Id: Ieee0470b6a2d07168a1578552e8e7f271ae93a13 * retrigger Change-Id: I753d63853171e6a436c104ce176048892eb974c5 * semantic merge conflict Change-Id: I9419448496de90340569047a6a16a1b858a7978a * update total to match prod defaults Change-Id: I693d55c1ebb28b5803ee1d26e9e198decf82308b * retrigger Change-Id: I28b74d5d6202f61aa3866fe407d423f6a0a14b9e * retrigger Change-Id: I6fd054885c715f602e2cef623fd464c42e88742c * retrigger Change-Id: I6a01bae88c72406d4ed5a8f13bf8a2b3c650bd2d
2019-06-27 18:36:51 +01:00
github.com/zeebo/structs v1.0.2
update dependencies for bug fixes (#1993)
2019-05-19 15:58:30 +01:00
go.uber.org/zap v1.10.0
go.mod: Update golang.org/x/crypto to fix vulnerability Update the golang.org/x/crypto package to fix the vulnerability CVE-2020-9283. See https://groups.google.com/forum/#!topic/golang-nuts/XDqhhjZViNk Change-Id: I7c841c0bae0f55dad0c7de19ac70c730d11733f0
2020-02-21 10:30:05 +00:00
golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975
go.mod: update to grpc 1.23.1 and update golang.org/x (#3072)
2019-09-19 13:41:19 +01:00
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e
pkg/process: Now that we are trying to identify the root cause of the satellite load limitations (i.e. currently the satellite has a max ability of 400 rps for uploads and we need this to be higher), we are using the golang diagnostic tools to collect insight into what the bottlenecks are. We currently have a debug endpoint to gather some cpu and mem data, but it could be useful to have continuous profiling. GCP stackdriver has support for continuous profiling so lets set that up and see if it is helpful to gather more data. This PR adds support for [GCP continuous profiler](https://cloud.google.com/profiler) which allows enabling continuous cpu/mem profiling and the stats are sent to stackdriver in google cloud console. To enable the continuous profiling for a storj component, do the following: - prereq: the workload must be running in GKE and have Stackdriver Profiling IAM role permissions - provide the config flag `debug.profilename` in the config.yaml file for the workload (i.e. satellite api process, etc). The profilename should be the workload name, for example "satellite-api". - once the above config flag is provided, the profiler will be initialized and profiling stats will automatically be sent to GCP project where the workload is running and viewable in the Stackdriver Profile page in the console The current implementation assumes the workload is running in GKE, however if we find if useful we can add support to enable this from anywhere. But for simplicity, its configured this way assuming the main goal is to enable in production systems. Change-Id: Ibf8ebe2df7bf06fdd4951ee6a1e48854dd36ad47
2020-02-25 16:46:12 +00:00
golang.org/x/sys v0.0.0-20200113162924-86b910548bc1
mod: bump common and some other packages Change-Id: Icf73220f5182f5a2fd1431471825948c300c0bea
2020-01-07 16:09:54 +00:00
golang.org/x/time v0.0.0-20191024005414-555d28b269f0
pkg/process: Now that we are trying to identify the root cause of the satellite load limitations (i.e. currently the satellite has a max ability of 400 rps for uploads and we need this to be higher), we are using the golang diagnostic tools to collect insight into what the bottlenecks are. We currently have a debug endpoint to gather some cpu and mem data, but it could be useful to have continuous profiling. GCP stackdriver has support for continuous profiling so lets set that up and see if it is helpful to gather more data. This PR adds support for [GCP continuous profiler](https://cloud.google.com/profiler) which allows enabling continuous cpu/mem profiling and the stats are sent to stackdriver in google cloud console. To enable the continuous profiling for a storj component, do the following: - prereq: the workload must be running in GKE and have Stackdriver Profiling IAM role permissions - provide the config flag `debug.profilename` in the config.yaml file for the workload (i.e. satellite api process, etc). The profilename should be the workload name, for example "satellite-api". - once the above config flag is provided, the profiler will be initialized and profiling stats will automatically be sent to GCP project where the workload is running and viewable in the Stackdriver Profile page in the console The current implementation assumes the workload is running in GKE, however if we find if useful we can add support to enable this from anywhere. But for simplicity, its configured this way assuming the main goal is to enable in production systems. Change-Id: Ibf8ebe2df7bf06fdd4951ee6a1e48854dd36ad47
2020-02-25 16:46:12 +00:00
google.golang.org/grpc v1.26.0
all: use monkit v3 this commit updates our monkit dependency to the v3 version where it outputs in an influx style. this makes discovery much easier as many tools are built to look at it this way. graphite and rothko will suffer some due to no longer being a tree based on dots. hopefully time will exist to update rothko to index based on the new metric format. it adds an influx output for the statreceiver so that we can write to influxdb v1 or v2 directly. Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2019-11-08 20:40:39 +00:00
gopkg.in/yaml.v2 v2.2.4
all: use pbgrpc common/pb moved grpc to a separate package common/pb/pbgrpc. This updates this repository to use it. Change-Id: I2de2a190688871cf9cb61f7ea511f8a01e264e4e
2020-02-26 13:59:46 +00:00
storj.io/common v0.0.0-20200226185531-a823b4a45116
all: use monkit v3 this commit updates our monkit dependency to the v3 version where it outputs in an influx style. this makes discovery much easier as many tools are built to look at it this way. graphite and rothko will suffer some due to no longer being a tree based on dots. hopefully time will exist to update rothko to index based on the new metric format. it adds an influx output for the statreceiver so that we can write to influxdb v1 or v2 directly. Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2019-11-08 20:40:39 +00:00
storj.io/drpc v0.0.8
cmd/gateway: simplify module handling Change-Id: If6ed158a6c9568fa33f69ca2d52e231ee4fcb0cb
2020-02-26 10:45:12 +00:00
storj.io/uplink v1.0.0-rc.2
Fix go.mod (#395)
2018-09-28 16:44:02 +01:00
)
Reference in New Issue Copy Permalink