storj/pkg/peertls/tlsfileoptions.go
Bryan White 5b913c45b9
Transport security (#63)
* wip initial transport security

* wip: transport security (add tests / refactor)

* wip tests

* refactoring - still wip

* refactor, improve tests

* wip tls testing

* fix typo

* wip testing

* wip testing

* wip

* tls_test passing

* code-style improvemente / refactor; service and tls tests passing!

* code-style auto-format

* add TestNewServer_LoadTLS

* refactor; test improvements

* refactor

* add client cert

* port changes

* Merge remote-tracking branch 'upstream/master'

* Merge remote-tracking branch 'upstream/master'

* Merge remote-tracking branch 'upstream/master'

* files created

* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests

* wip

* add separate `Process` tests for bolt and redis-backed overlay

* more testing

* fix gitignore

* fix linter error

* goimports goimports GOIMPORTS GoImPortS!!!!

* wip

* fix port madness

* forgot to add

* add `mux` as handler and shorten context timeouts

* gofreakingimports

* fix comments

* refactor test & add logger/monkit registry

* debugging travis

* add comment

* Set redisAddress to empty string for bolt-test

* Merge remote-tracking branch 'upstream/master' into coyle/kad-tests

* Merge branch 'tls' into tls-upstream

* tls:
  add client cert
  refactor
  refactor; test improvements
  add TestNewServer_LoadTLS
  code-style auto-format
  code-style improvemente / refactor; service and tls tests passing!
  tls_test passing
  wip
  wip testing
  wip testing
  fix typo
  wip tls testing
  refactor, improve tests
  refactoring - still wip
  wip tests
  wip: transport security (add tests / refactor)
  wip initial transport security

* fixing linter things

* wip

* remove bkad dependencie from tests

* wip

* wip

* wip

* wip

* wip

* updated coyle/kademlia

* wip

* cleanup

* ports

* overlay upgraded

* linter fixes

* piecestore kademlia newID

* Merge branch 'master' into tls-upstream

* master:
  Add error to the return values of Ranger.Range method (#90)
  udp-forwarding: demo week work! (#84)

* Merge branch 'kad-tests' into tls-upstream

* kad-tests:
  piecestore kademlia newID
  linter fixes
  overlay upgraded
  ports
  cleanup
  wip
  updated coyle/kademlia
  wip
  wip
  wip
  wip
  wip
  remove bkad dependencie from tests
  wip
  wip
  files created
  port changes

* wip

* finish merging service tests

* add test for different client/server certs

* wip

* Merge branch 'master' into tls-upstream

* master:
  Add context to Ranger.Range method (#99)
  Coyle/kad client (#91)

* wip

* wip; refactoring/cleanup

* wip

* Merge branch 'master' into tls

* master:
  Bolt backed overlay cache (#94)
  internal/test: switch errors to error classes (#96)

* wip - test passing

* cleanup

* remove port.go

* cleanup

* Merge branch 'master' into tls

* master:
  hardcode version (#111)
  Coyle/docker fix (#109)
  pkg/kademlia tests and restructuring (#97)
  Use continue instead of return in table tests (#106)
  prepend storjlabs to docker tag (#108)
  Automatically build, tag and push docker images on merge to master (#103)

* more belated merging

* more belated merging

* more belated merging

* add copyrights

* cleanup

* goimports

* refactoring

* wip

* wip

* implement `TLSFileOptions#loadTLS`, refactoring:

`peertls.TestNewClient_LoadTLS` is the failing holdout; Still trying to figure out why I'm getting ECDSA verification is failing.

* not sure if actually working:

Tests are now passing (no more "ECDSA verification failed"); however,
`len(*tls.Certificates.Certificate) == 1` which I don't think should be
the case if the root and leaf are being created correctly.

* Experimenting/bugfixing?:

I think leaf certs should be properly signed by the parent now but not
entirely certain. It's also unclear to me why in
`VerifyPeerCertificate`, `len(rawCerts) == 1` when the certs should
contain both the root and leaf afaik.

* Properly write/read certificate chain (root/leaf):

I think I'm now properly reading and writing the root and leaf
certificate chain such that they're both being received by
`VerifyPeerCertificate`.

The next step is to parse the certificates with `x509.ParseCertificate`
(or similar) and verify that the public keys and signatures match.

* Add tls certificate chain signature veification (spike):

+ `VerifyPeerCertificate` verifies signatures of certificates using the
key of it's parent if there is one; otherwise, it verifies the
certificate is self-signed
+ TODO: refactor
+ TODO: test

* refactoring `VerifyPeerCertificate`

* cleanup

* refactor

* Merge branch 'master' into tls

* master:
  Remove some structural folders we don't seem to be using. (#125)
  license code with agplv3 (#126)
  Update .clabot (#124)
  added team memebers (#123)
  clabot file added (#121)
  ECClient (#110)
  docker image issue fixed (#118)
  Piecestore Farmer CLI  (#92)
  Define Path type (#101)
  adds netstate pagination (#95)
  Transport Client (#89)
  Implement psclient interface (#107)
  pkg/process: start replacing pkg/process with cobra helpers (#98)
  protos/netstate: remove stuff we're not using (#100)
  adding coveralls / code coverage  (#112)

* responding to review feedback / cleanup / add copywrite headers

* suggestions

* realitive

* Merge pull request #1 from coyle/coyle/tls

suggestions

* remove unnecessary `_`s

* Merge branch 'tls' of github.com:bryanchriswhite/storj into tls

* 'tls' of github.com:bryanchriswhite/storj:
  realitive
  suggestions

* Responding to review feedback:

+ refactor `VerifyPeerCertificate`

* remove tls expiration

* remove "hosts" and "clien option" from tls options

* goimports

* linter fixes
2018-07-09 19:43:13 +02:00

203 lines
4.5 KiB
Go

// Copyright (C) 2018 Storj Labs, Inc.
// See LICENSE for copying information.
package peertls
import (
"crypto/tls"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/zeebo/errs"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
)
type fileRole int
const (
rootCert fileRole = iota
rootKey
leafCert
leafKey
)
var (
fileLabels = map[fileRole]string{
rootCert: "root certificate",
rootKey: "root key",
leafCert: "leaf certificate",
leafKey: "leaf key",
}
)
// EnsureAbsPath ensures that the absolute path fields are not empty, deriving them from the relative paths if not
func (t *TLSFileOptions) EnsureAbsPaths() error {
for _, role := range t.requiredFiles() {
for absPtr, relPath := range t.pathMap() {
if t.pathRoleMap()[absPtr] == role {
if *absPtr == "" {
if relPath == "" {
return ErrTLSOptions.New("no relative %s path provided", fileLabels[t.pathRoleMap()[absPtr]])
}
absPath, err := filepath.Abs(relPath)
if err != nil {
return ErrTLSOptions.Wrap(err)
}
*absPtr = absPath
}
}
}
}
return nil
}
// EnsureExists checks whether the cert/key exists and whether to create/overwrite them given `t`s field values.
func (t *TLSFileOptions) EnsureExists() error {
if err := t.EnsureAbsPaths(); err != nil {
return err
}
hasRequiredFiles, err := t.hasRequiredFiles()
if err != nil {
return err
}
if t.Create && !t.Overwrite && hasRequiredFiles {
return ErrNoOverwrite.New("certificates and keys exist; refusing to create without overwrite")
}
// NB: even when `overwrite` is false, this WILL overwrite
// a key if the cert is missing (vice versa)
if t.Create && (t.Overwrite || !hasRequiredFiles) {
if err := t.generateTLS(); err != nil {
return err
}
return nil
}
if !hasRequiredFiles {
missing, _ := t.missingFiles()
return ErrNotExist.New(fmt.Sprintf(strings.Join(missing, ", ")))
}
// NB: ensure `t.Certificate` is not nil when create is false
if !t.Create {
t.loadTLS()
}
return nil
}
func (t *TLSFileOptions) NewTLSConfig(c *tls.Config) *tls.Config {
config := cloneTLSConfig(c)
config.Certificates = []tls.Certificate{*t.LeafCertificate}
// Skip normal verification
config.InsecureSkipVerify = true
// Required client certificate
config.ClientAuth = tls.RequireAnyClientCert
// Custom verification logic for *both* client and server
config.VerifyPeerCertificate = VerifyPeerCertificate
return config
}
func (t *TLSFileOptions) NewPeerTLS(config *tls.Config) credentials.TransportCredentials {
return credentials.NewTLS(t.NewTLSConfig(config))
}
func (t *TLSFileOptions) DialOption() grpc.DialOption {
return grpc.WithTransportCredentials(t.NewPeerTLS(nil))
}
func (t *TLSFileOptions) ServerOption() grpc.ServerOption {
return grpc.Creds(t.NewPeerTLS(nil))
}
func (t *TLSFileOptions) loadTLS() (_ error) {
leafC, err := LoadCert(t.LeafCertAbsPath, t.LeafKeyAbsPath)
if err != nil {
return err
}
t.LeafCertificate = leafC
return nil
}
func (t *TLSFileOptions) missingFiles() ([]string, error) {
missingFiles := []string{}
paths := map[fileRole]string{
rootCert: t.RootCertAbsPath,
rootKey: t.RootKeyAbsPath,
leafCert: t.LeafCertAbsPath,
leafKey: t.LeafKeyAbsPath,
}
requiredFiles := t.requiredFiles()
for _, requiredRole := range requiredFiles {
for role, path := range paths {
if role == requiredRole {
if _, err := os.Stat(path); err != nil {
if !IsNotExist(err) {
return nil, errs.Wrap(err)
}
missingFiles = append(missingFiles, fileLabels[role])
}
}
}
}
return missingFiles, nil
}
func (t *TLSFileOptions) requiredFiles() []fileRole {
var roles = []fileRole{}
// rootCert is always required
roles = append(roles, rootCert, leafCert, leafKey)
if t.Create {
// required for writing rootKey when create is true
roles = append(roles, rootKey)
}
return roles
}
func (t *TLSFileOptions) hasRequiredFiles() (bool, error) {
missingFiles, err := t.missingFiles()
if err != nil {
return false, err
}
return len(missingFiles) == 0, nil
}
func (t *TLSFileOptions) pathMap() map[*string]string {
return map[*string]string{
&t.RootCertAbsPath: t.RootCertRelPath,
&t.RootKeyAbsPath: t.RootKeyRelPath,
&t.LeafCertAbsPath: t.LeafCertRelPath,
&t.LeafKeyAbsPath: t.LeafKeyRelPath,
}
}
func (t *TLSFileOptions) pathRoleMap() map[*string]fileRole {
return map[*string]fileRole{
&t.RootCertAbsPath: rootCert,
&t.RootKeyAbsPath: rootKey,
&t.LeafCertAbsPath: leafCert,
&t.LeafKeyAbsPath: leafKey,
}
}