storj/pkg/piecestore/rpc/client/client.go

160 lines
4.5 KiB
Go
Raw Normal View History

Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
// Copyright (C) 2018 Storj Labs, Inc.
// See LICENSE for copying information.
package client
import (
"bufio"
"flag"
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
"fmt"
"io"
"log"
"time"
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
"github.com/zeebo/errs"
"go.uber.org/zap"
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
"golang.org/x/net/context"
"google.golang.org/grpc"
"storj.io/storj/pkg/ranger"
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
pb "storj.io/storj/protos/piecestore"
)
// ClientError is any error returned by the client
var ClientError = errs.Class("PSClient error")
var (
defaultBandwidthMsgSize = flag.Int(
"piecestore.rpc.client.default_bandwidth_msg_size", 32*1024,
"default bandwidth message size in kilobytes")
maxBandwidthMsgSize = flag.Int(
"piecestore.rpc.client.max_bandwidth_msg_size", 64*1024,
"max bandwidth message size in kilobytes")
)
// PSClient is an interface describing the functions for interacting with piecestore nodes
type PSClient interface {
Meta(ctx context.Context, id PieceID) (*pb.PieceSummary, error)
Put(ctx context.Context, id PieceID, data io.Reader, ttl time.Time, ba *pb.PayerBandwidthAllocation) error
Get(ctx context.Context, id PieceID, size int64, ba *pb.PayerBandwidthAllocation) (ranger.RangeCloser, error)
Delete(ctx context.Context, pieceID PieceID) error
CloseConn() error
}
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
// Client -- Struct Info needed for protobuf api calls
type Client struct {
route pb.PieceStoreRoutesClient
conn *grpc.ClientConn
pkey []byte
bandwidthMsgSize int
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
}
// NewPSClient initilizes a PSClient
func NewPSClient(conn *grpc.ClientConn, bandwidthMsgSize int) (PSClient, error) {
if bandwidthMsgSize < 0 || bandwidthMsgSize > *maxBandwidthMsgSize {
return nil, ClientError.New(fmt.Sprintf("Invalid Bandwidth Message Size: %v", bandwidthMsgSize))
}
if bandwidthMsgSize == 0 {
bandwidthMsgSize = *defaultBandwidthMsgSize
}
return &Client{
conn: conn,
route: pb.NewPieceStoreRoutesClient(conn),
bandwidthMsgSize: bandwidthMsgSize,
}, nil
2018-06-03 05:20:17 +01:00
}
// NewCustomRoute creates new Client with custom route interface
func NewCustomRoute(route pb.PieceStoreRoutesClient, bandwidthMsgSize int) (*Client, error) {
if bandwidthMsgSize < 0 || bandwidthMsgSize > *maxBandwidthMsgSize {
return nil, ClientError.New(fmt.Sprintf("Invalid Bandwidth Message Size: %v", bandwidthMsgSize))
}
if bandwidthMsgSize == 0 {
bandwidthMsgSize = *defaultBandwidthMsgSize
}
return &Client{
route: route,
bandwidthMsgSize: bandwidthMsgSize,
}, nil
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
}
// CloseConn closes the connection with piecestore
func (client *Client) CloseConn() error {
return client.conn.Close()
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
}
// Meta requests info about a piece by Id
func (client *Client) Meta(ctx context.Context, id PieceID) (*pb.PieceSummary, error) {
return client.route.Piece(ctx, &pb.PieceId{Id: id.String()})
}
// Put uploads a Piece to a piece store Server
func (client *Client) Put(ctx context.Context, id PieceID, data io.Reader, ttl time.Time, ba *pb.PayerBandwidthAllocation) error {
stream, err := client.route.Store(ctx)
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
if err != nil {
return err
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
}
msg := &pb.PieceStore{Piecedata: &pb.PieceStore_PieceData{Id: id.String(), ExpirationUnixSec: ttl.Unix()}}
if err = stream.Send(msg); err != nil {
if _, closeErr := stream.CloseAndRecv(); closeErr != nil {
zap.S().Errorf("error closing stream %s :: %v.Send() = %v", closeErr, stream, closeErr)
}
return fmt.Errorf("%v.Send() = %v", stream, err)
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
}
writer := &StreamWriter{signer: client, stream: stream, pba: ba}
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
defer func() {
if err := writer.Close(); err != nil && err != io.EOF {
log.Printf("failed to close writer: %s\n", err)
}
}()
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
bufw := bufio.NewWriterSize(writer, 32*1024)
_, err = io.Copy(bufw, data)
if err == io.ErrUnexpectedEOF {
writer.Close()
zap.S().Infof("Node cut from upload due to slow connection. Deleting piece %s...", id)
return client.Delete(ctx, id)
}
if err != nil {
return err
}
return bufw.Flush()
}
// Get begins downloading a Piece from a piece store Server
func (client *Client) Get(ctx context.Context, id PieceID, size int64, ba *pb.PayerBandwidthAllocation) (ranger.RangeCloser, error) {
stream, err := client.route.Retrieve(ctx)
if err != nil {
return nil, err
}
return PieceRangerSize(client, stream, id, size, ba), nil
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
}
// Delete a Piece from a piece store Server
func (client *Client) Delete(ctx context.Context, id PieceID) error {
reply, err := client.route.Delete(ctx, &pb.PieceDelete{Id: id.String()})
Added rpc server (#40) * Added piecestore * gofmt * Added requested changes * Added cli * Removed ranger because I wanted something that can stand alone * Add example of http server using piece store * Changed piecestore code to make it more optial for error handelling * Merged with piecestore * Added missing package * Forgot io import * gofmt * gofmt * Forgot io * Make path by hash exported * updated to simplify again whoops * Updated server to work real good * Forgot ampersand * Updated to match FilePiece * Merged in cam's delete code * Remove unused io * Added RPC code * Give the download request a reader * Removed http server stuff; changed receive stream to say io.reader * Added expiration date to shardInfo * Change all instances of Shard to Piece; change protobuf name; moved client insance to outside functions * added ttl info request * Move scripts to http server pr; added close method for Retrieve api * added rpc server tests for getting piece meta data and retrieval routes * Resolved linter errors, moved to prc server to pkg, updated go.mod to use latest protobuf * Imported cams test * Bump gometalinter deadline * WIP adding tests * added tests for store and delete routes * Add changes as requested by Kaloyan, also cleaned up some code * Get the code actually working whoops * More cleanup * Separating database calls from api.go * need to rename expiration * Added some changes requested by JT * Fix read size * Fixed total amount to read * added tests * Simplify protobuf, add store tests, edited api to handle invalid stores properly, return errors instead of messages * Moved rpc client and server to piece store * Moved piecestore protobuf to the /protos folder * Cleaned up messages * Clean up linter errors * Added missing sqlite import * Add ability to do iterative reads and writes to pstore * Incrementally read data * Fix linter and import errors * Solve linter Error * Change return types * begin test refactor * refactored to implement only 1 db connection, moved SQLite row checking into separate function and removed defer on rows.Close(), fixed os.tempDir in rpc_test.go * Cleaning up tests * Added retrieve tests * refactored delete tests * Deleted old tests * Updated cmd/piecestore to reflect changes to piecestore * Refactored server tests and server/client store code * gofmt * WIP implementing TTL struct * Read 4k at a time when Retrieving * implemented ttl struct * Accidentally removed fpiece dependency? * Double resolve merge conflict * Moved client to the examples since it is an example * Change hash to id in protobuf. TODO: Change client and server to reflect these changes * Update client and protobuf * changed hash to id * Handle eof properly in retrieve * Id -> ID * forgot to change import for client after moving * Moved client and server main to examples * Made changes requested by JT * checkEntries is now private, created currentTime variable for checkEntries, added defer rows.Close() * Print not fatal * Handle overflow when reading from server * added const IDLength * removed types from comments * Add reader/writer for download data from and uploading data to server * goimports and comments * fixed nits, casts, added OK const, DBCleanup now exits program on error * Add stream reader and writer for server * Fix errors * i beforee except after c lol * customizable data dir * Forgot to rename variable * customizable data dir * Handle closing of server stream properly * linter * pass on inserting the same data twice * linter * linter * Do the easy things JT asked for * Handle overflow from reads properly; handle custom db path * Handle overflow for server stream read; TODO Combine server and client stream reads * Allow for TTL of 0 to stay forever * Make Client cleaner for user
2018-06-02 19:14:59 +01:00
if err != nil {
return err
}
log.Printf("Route summary : %v", reply)
return nil
}
// sign a message using the clients private key
func (client *Client) sign(msg []byte) (signature []byte, err error) {
// use c.pkey to sign msg
return signature, err
}