Sia Coin November 2016 Update + Roadmap
- Successful v1.0.3 release - very stable.
- v1.0.4-LTS announced
- v1.1.0 wallet upgrades, including instant wallet unlock, slated for mid-December 2016
- v1.2.0 host upgrades, including pricing support, slated for early January 2017
- v1.3.0 renter upgrades, including speed and scalability boosts, slated for late January 2017
- File sharing added back to the medium-term roadmap, estimated March - June 2017.
- Sia now has an active social media + messaging team
- The infographic bounty remains live
- Sia is now actively doing joint-events with other communities in the blockchain and data storage space
The previous month saw the release of v1.0.3, the most stable and usable release we've had to date, by a long shot. Despite this, a few bugs were discovered after release by our users. Within the next week or so we'll be releasing another version, v1.0.4 with minor fixes to some small-and-rare problems that occur in v1.0.3.
Our primary focus however has shifted from testing + bugfixing to feature development. We're entering a 3-6 month cycle of upgrades, including instant wallet unlocking, host performance improvements, Sybil attack resistance, and renter performance improvements.
We're dividing our releases into two major categories. There's the standard line of releases, and then there's the LTS line of releases. The standard line of releases will match what we've already been doing, frequent versions with new features, bug fixes, and security upgrades all rolled together. The LTS line of releases will not have any of the new features, but will have as many of the bugfixes and security patches as possible. This can be attractive for stability, as new features can introduce new bugs, and frequent upgrades can cause frequent headaches.
v1.0.4 is going to be our first LTS release, and we promise to support it until the next LTS release is available. This means that sometimes we could end up releasing a v1.0.5 and a v1.1.0 at the same time, if we discovered bugs in v1.0.4 that we felt needed to be addressed. v1.0.5 would have the most important bugfixes, but none of the new features.
v1.1.0 is slated to have major wallet upgrades, and is targeted for mid-December. We've got a github project open explaining everything that we are hoping to achieve, you can check it out here: https://github.com/NebulousLabs/Sia/projects/1
The most significant change is that the wallet will now unlock in under a second. This can be compared to the multi-minute unlock times of the current wallet.
Today, when you load a seed into the wallet, it does not always find all of your money. Many users have run into this problem, and while we've been able to resolve the issues for everyone, it really should find all of your money on the first try. The new wallet does exactly that - it's more thorough and complete when it scans the blockchain, meaning that it's not going to miss money anymore.
Today, when you load a seed or a key into the wallet, you have to restart Sia and then re-unlock your wallet for it to be able to find the new money. This is also no longer the case. Loading seeds + keys will take some time, as a blockchain rescan has to be performed, however you do not need to restart Sia.
Currently if you want to back up your whole wallet, you have to open up a file browser and navigate to the wallet folder. The new wallet will have a simple 'create encrypted backup' process/button that allows you to do it without needing to know where Sia puts the actual data. This will be accompanied by a 'restore encrypted backup' process/button that will allow you to restore your wallet using a backup.
You will no longer need to create a new wallet to load a seed you've previously used. You'll be able to create the new wallet using the seed, meaning you don't have to add another seed to the list of seeds you are keeping.
You will also be able to 'sweep' keys and seeds, meaning that it's much easier to consolidate all of your funds into a single seed. You will also be able to sweep keys and seeds individually, meaning you can clean out one set of seeds (perhaps your hot wallet) without cleaning up your other set (perhaps the cold wallet).
There are few other upgrades to the wallet coming as well. @nemo is the primary developer on this project.
v1.2.0 is slated to have major host upgrades, and is targeted for early-January, just 2-3 weeks after v1.1.0. We're able to have a tight turnaround on the releases because we will be working on the host upgrades and wallet upgrades in parallel. Wallet upgrades are slated to be released first, followed by the host upgrades.
The host right now can realistically only handle a few hundred GBs of data. After that, performance starts to degrade and the host begins to put a heavy burden on your filesystem. It's actually dangerous for renters to store more than a few hundred GBs on a single host, because file contract renew operations will begin taking longer than the timeout allows, putting the renter at high risk of losing files when a renew happens. Additionally, you currently need to run the host as admin on Windows if you want to be able to add storage folders, which is a bit of a silly requirement. Furthermore, the fundamental design of the host means that small amounts of data occasionally get lost, and small amounts of data occasionally get added accidentally, in a way that can never be removed. Hosts running for years under the current code would eventually fill up with 'useless' data, noticably hurting overall profitability.
The new host has been designed from the ground up with performance and stability in mind. It's already about 50% complete, and it's fast, and it scales. You aren't likely to notice unless you're storing hundreds of GBs of data, but where the current host seems to top out below 1 TB, the new host can in theory support well above 100 TiB per machine. We have limited testing in place already to simulate loads that high, and so far it's held up just fine.
In addition to speed and scale, the new host features a more robust ACID architecture, meaning it's much less likely that data gets lost or gets stuck.
The host is also (finally) going to have tools that allow you to inspect the contracts that you have, so you can see how big each are, how much money is coming in, how close they are to finalizing on the blockchain, etc.
We're also hoping to release tools that will help hosts set the price more intelligently. We recognize that it's currently a process of stubmling around in the dark until you find money, and these tools will provide stronger feedback about how the host is doing and what steps can be taken to boost visibility + revenue. This is doubly important now because renters no longer only look at the storage price of the host. They also look at uptime, version (more recent versions are favored), download price, and a few other metrics that affect your host.
Shortly following the v1.2.0 release will be guide explaining how to use the new host most effectively.
The next round of renter upgrades is slated for late January 2017, shortly following the host upgrades. Changes to the host will allow us to make significant improvements to the renter, including substantial speed and scale upgrades. The major focus is going to be on getting the speed to a point where it can properly saturate your home connection for both uploads and downloads. A secondary focus will be on boosting the total file size supported by the renter. Currently the renter will start to struggle beyond about 100 GB, and we're hoping to jump the new renter to be able to support about 10 TB of data. Early designs are being discussed, however the substantial work has not started. This can be contrasted with the host and wallet, where most of the heavy lifting is already complete. At this point in time, we predict that it will be safe to reduce the default redundancy in the renter from 6x to 4x, meaning uploads will be faster (less total data to upload), and storage will be cheaper (less data to store).
Currently the API only allows you to interact directly with the renter, meaning that if you want to make an app that automatically handles contracts, redundancy, etc., you have to write in golang and use our renter interfaces (which don't have any compatibility promises). The new renter will change this, providing API endpoints for working with the contractor, meaning you can pick your own hosts, decide your own contract parameters, and ultimately have a lot more freedom to customize Sia to your specific needs.
Long Term Upgrades
We have added file sharing back to the roadmap. Previously, we could not commit to supporting file sharing because we didn't know how to make the micropayments work. On Sia, all downloads cost money, and that includes downloading files that are shared with you. Typically, you can use the file contracts you have with a host to pay for the downloads, meaning you don't need to add any transactions to the blockchain or impact scalability in any way. You can (and do!) make thousands of tiny payments to your hosts for the data you download, and this is perfectly scalable. With file sharing however, you often don't have file contracts with the hosts storing the file. This is a problem, because it means you don't have an easy way to do micropayments to them. Either you pay an absurd amount in transaction fees + bloat the blockchain, or... ???... basically it was an unsolved problem.
We've since solved the problem. We will be able to set up a lightning-network style set of payment channels between the hosts. When you download a file from an unknown host, you can pay your host, then your host will pay the unknown host, and you can download the files without using micropayments, and without bloating the blockchain. We figured out a few tricks we can apply to Sia that makes this a much simpler problem to solve than building out an entire Bitcoin-style lightning network, meaning it should only take us a few weeks to get a prototype working (and then another few weeks to get it to production-grade).
We still need to finish the renter and host upgrades before we can start offering file sharing, so file sharing remains a ways off, but it no longer has unsolved problems preventing us from committing to it.
For the past year or more, Sia has kept most everything internal, posting primarily to the Sia forum and hanging out primarily in the Sia slack. This is a good strategy when you are new and vulnerable, however Sia is now a more mature project, especially for the blockchain ecosystem. We're starting initiatives to get Sia integrated with other communities in the blockchain ecosystem, especially communities that already have membership overlap or otherwise share ideals with Sia.
We kicked this off with an AMA in the Cryptocopia slack, and are following up with participation in the BEN Gauntlet. We hope to continue this trend by participating in more events with other communities, and helping to add to the global decentralization movement that's been gaining traction over the past few years. If you have ideas, please don't hesitate to reach out.
You may have noticed that we've revitalized some of our social media efforts. Our Twitter account is more active, the forum is more active, and our subreddit is more active. This is because we now have small team actively managing our social media and messaging.
The infographic bounty is still live. We've thus far only had a few submissions, however we are still accepting submissions on a rolling basis and will be paying out bounties to all quality submissions.