How do I manually write an EC stream?

Owned by Aaron Enfield
Jun 29, 2017
1 min read

Writing EC requires a minimum parity.

Assuming you have a default encoding value (k:p) configured in the cluster, you do not have to specify the encoding in your write.  Otherwise, you do need to specify the encoding.  Assuming your stream is larger than the minimum stream size required to be encoded, you will then either pick up the default encoding or apply the encoding from the write directive.

For example:

[root@b-csn5 ~]# ls -lh tmp.file
-rw-r--r-- 1 root root 5.0M Jan 28 17:16 tmp.file

With encoding:

[root@b-csn5 ~]# curl -L -i --post301 --data-binary @tmp.file "http://b-csn5.enfield.com?encoding=4:2"
HTTP/1.1 201 Created
Volume: c603ca71bc4cea70a39029ec51176a70
Volume: 02f82452f3cf647668d4fa173880e17e
Volume: 017037ff881430838aa71529ba912162
Manifest: ec
Content-UUID: 1fd487f2bd4a328c331dfbf3b8ac4ff2
Last-Modified: Tue, 28 Jan 2014 23:26:16 GMT
Entity-MD5: Oo3s3hRNgTkOlN5Tb9xWDg==
Stored-Digest: 3a8decde144d81390e94de536fdc560e
Castor-System-Encoding: zfec 1.4(4, 2, 524288, 200000000)
Castor-System-Version: 1390951576.562
Etag: "5b70c878b2ddf185af9ed45b288ae179"
Location: http://10.1.1.165:80/1fd487f2bd4a328c331dfbf3b8ac4ff2
Via: 1.1 b-csn5 (ScspProxyService/6.1.1)
X-Forwarded-For: b-csn5
X-Forwarded-Server: b-csn5
Scsp-Proxy-Agent: ScspProxyService/6.1.1
Date: Tue, 28 Jan 2014 23:26:16 GMT
Server: CAStor Cluster/6.1.3
Content-Length: 46
Content-Type: text/html
<html><body>New stream created</body></html>

You can see where the individual segments are by doing a READ on the manifest.

[root@b-csn5 ~]# curl -Li "http://b-csn5.enfield.com/1fd487f2bd4a328c331dfbf3b8ac4ff2?checkIntegrity"
HTTP/1.1 200 OK
Castor-System-Cluster: b-csn5.enfield.com
Castor-System-Created: Tue, 28 Jan 2014 23:26:16 GMT
Castor-System-Version: 1390951576.562
Content-Type: application/x-www-form-urlencoded
Last-Modified: Tue, 28 Jan 2014 23:26:16 GMT
Content-Length: 966
Cache-Control: no-cache
Etag: "5b70c878b2ddf185af9ed45b288ae179"
Manifest: ec
Volume: c603ca71bc4cea70a39029ec51176a70
Via: 1.1 b-csn5 (ScspProxyService/6.1.1)
X-Forwarded-For: b-csn5
X-Forwarded-Server: b-csn5
Scsp-Proxy-Agent: ScspProxyService/6.1.1
Date: Tue, 28 Jan 2014 23:27:09 GMT
Server: CAStor Cluster/6.1.3
set:0 segment:35368d20b3f2f05fcc98c9e60396f9cf size:1310724 reps:1 loc:192.168.5.84 vol:01b8f89cbb79777b304f936531af7d23 sibVol:3848be9920526557da30469a1abdb9d3
set:0 segment:49a1aeebebd878b9344776ac9abadae2 size:1310724 reps:1 loc:192.168.5.85 vol:3848be9920526557da30469a1abdb9d3 sibVol:2994876ed85804a5f9b88a78f8ade9ba
set:0 segment:26094b5f079d86caa9a60b15de692126 size:1310724 reps:1 loc:192.168.5.86 vol:2994876ed85804a5f9b88a78f8ade9ba sibVol:e746d1987ca0b3442dffecd31287edd4
set:0 segment:9102a39def581de702c48b9561cd20c3 size:1310724 reps:1 loc:192.168.5.87 vol:e746d1987ca0b3442dffecd31287edd4 sibVol:c603ca71bc4cea70a39029ec51176a70
set:0 segment:138ddc9c83ebb708d9fdafcd95d00987 size:1310724 reps:1 loc:192.168.5.88 vol:c603ca71bc4cea70a39029ec51176a70 sibVol:6a381e0c8f2ea0b9cd82d3711aea8976
set:0 segment:ab370f61170909ea91c1f2a69ff84001 size:1310724 reps:1 loc:192.168.5.89 vol:6a381e0c8f2ea0b9cd82d3711aea8976 sibVol:01b8f89cbb79777b304f936531af7d23

© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.