Maidsafe coin Details+Update





  • SAFE Network Introduction Video:




    Features of the SAFE Network

    The SAFE (Secure Access For Everyone) Network is made up of the unused hard drive space, processing power and data connection of its users. It offers a level of security and privacy not currently available on the existing Internet and turns the tables on companies, putting users in control of their data, rather than trusting it to organisations. A number of features make this possible:


    Login and secure your data with no middle man.






    No need to give your password to anyone, or ask a third party’s permission to access your data. Your information, and access to it, belongs to you!

    Your PIN and Keyword are used to locate your data on the network and your password, which never leaves your machine, is then used to decrypt it locally. This means that no one needs to hold a record of your files, or your login details and there’s no need to ask anyone for permission to access it. This process is called Self-Authentication and enables you to find, unlock and decrypt your own data.



    Self-Encryption: Data which encrypts itself, with itself.





    Files uploaded to the network are broken into pieces, encrypted and distributed across the network. This process is called Self-Encryption.

    When a user uploads (or saves) a file to the network, via one of the SAFE Network apps, the file is automatically broken up into chunks. These chunks are then encrypted (encoded so that only authorised parties can read it), randomised and stored on the computers of other SAFE Network users. These encrypted chunks are completely unreadable and inaccessible to anyone other than the owner.


    Distributed network with opportunistic data caching.




    The SAFE Network is fully decentralised, with files distributed and stored all over the world, on different devices. This allows the network to be robust to attacks, with no central point of weakness.

    The distributed nature of the network provides your data with physical security, meaning that no third party can access or delete it, as can happen with existing centralised solutions. The feature called Opportunistic Caching automatically creates more copies of popular data closer to where it is being requested, so popular websites and other data feeds will actually speed up as they get more visitors, rather than slow down and crash as they do on today’s web.


    Data availability and built-in redundancy.





    The network is programmed to keep duplicate copies of each piece of data at all times. As users turn their computers off, the network makes more copies and stores them on other machines, ensuring that users always have access to their files.

    This constant movement of data (called churn) is a key part of the security that the SAFE Network offers because there is no central point for hackers to target as the data locations keep changing.


    Unneeded duplicates are automatically removed.



    Once a file is uploaded, other users who upload the same file will be automatically referred to the original, limiting the number of copies and reducing the computing resources needed to store it


    Crowd sourced internet:


    Rather than using data centres and servers common within today’s internet, which are prone to data theft and surveillance, the SAFE Network uses advanced P2P technology, that joins together the spare computing capacity of all SAFE users, creating a global network. You can think of SAFE as a crowd sourced internet. It is on this network that everyone’s data and applications reside.

    All you need to join this network is to download the free SAFE software, create your account and tell the network how much space you are going to provide. It’s that simple!


    Safecoin:


    Global distribution without human intervention.

    Safecoins are distributed entirely by the algorithms within the SAFE Network. Human beings are not involved in the process. Only 4.3 billion coins will ever be in circulation at one time and each will have its own unique identity. Safecoins will be recycled when users exchange them for network services, ensuring there is always a supply for users to earn


    Farming: an incentive for a crowd sourced Internet.




    Safecoins are given as an incentive to users for providing their resource to the network. This resource is their: storage space, CPU, bandwidth and online time that enable the encrypted chunks of network data to be stored and retrieved from their computer. The process of providing resource and receiving safecoin in return is called ‘Farming’. Each piece of encrypted network data is stored in a Farmer's ‘Vault’, a data storage and management location on the Farmer's computer which they cannot read, or access. The SAFE network is designed to self-manage these resources


    Resource based economy




    As network data is retrieved from a computer, the network pays safecoin to the user. This payment doesn’t take place every time data is retrieved, instead it is more like a lottery, where the Farmer is rewarded at random. The safecoin is automatically paid by the network into the user’s safecoin wallet address that is tied (cryptographically) to their vault. The amount of safecoin a user can earn is directly linked to how much resource they provide to the network and how often their computer is available to the network (turned on).



    Proof of resource.





    The resource provided by each Farmer is continually checked by the network to make sure that it is still available. It does this by attempting to store a random piece of data. Should the network find that the resource committed to the network is no longer available, it reduces the rank of the vault thereby reducing the earning potential of that specific Vault.


    A new market place.






    Safecoins can be used to access and use network applications, with part of the ‘payment’ going directly to the application developer. Safecoins can also be bought by users unwilling, or unable to provide resource to the network. This creates a market of buyers and sellers so that farmers can sell the coins they earn for another currency. These transactions will take place on an exchange, another SAFE Network application, or directly between individuals.



    Decentralised Applications





    The ecosystem around the SAFE Network is already starting to blossom and a number of exciting apps* are already being built by third party developers using the SAFE APIs.

    Applications that provide access via the Launcher API ensure that you can access all apps with the same PIN, username and password used when you created your SAFE account. This means that you stay in control of your details at all times.

    *The projects listed within the link above are built and maintained by independent parties.


    Team: https://maidsafe.net/company.html

    Download: https://maidsafe.net/alpha.html





  • safe_vault

    safe_launcher

    safe_core

    routing

    self_encryption

    https://github.com/maidsafe

    Links:


    [ref Network] MaidSafe Network website : www.maidsafe.net

    [ref Autonomous] Autonomous Network, David Irvine, Fraser Hutchison, Steve Mucklisch : https://github.com/maidsafe/MaidSafe/wiki/unpublished_papers/AutonomousNetwork.pdf?raw=true

    [ref Routing] MaidSafe Routing github site : https://github.com/maidsafe/MaidSafe-Routing/wiki

    [ref BitCoin] Bitcoin : A Peer-to-Peer Electronic Cash System, Satoshi Nakamoto, https://bitcoin.org/bitcoin.pdf‎

    [ref RUDP] MaidSafe RUDP github site : https://github.com/maidsafe/MaidSafe-RUDP/wiki

    [ref Persona] Vault Documentation : https://github.com/maidsafe/MaidSafe-Vault/wiki/Documentation

    [ref Escrow] The Escrow service for Bitcoin : http://btcrow.com/

    White Paper:https://github.com/maidsafe/Whitepapers/blob/maste...



  • MaidSafe Announces Equity Funding Round on BnktotheFuture

    http://cryptocentral.info/topic/430/maidsafe-annou...






  • RFC 42 – SAFE Launcher API v0.6

    https://github.com/maidsafe/rfcs/blob/master/text/...




    Summary

    Expose low level API from launcher to allow applications to allow thrid party applications to access the building blocks of the SAFE Network.

    Motivation

    Exposing low level apis allow dynamic applications to be built on SAFE Network. Using the StructuredData API third party applications can build and manage their own data structures to fit their needs.

    Detailed design

    Expose low level building blocks via launcher for third party applications, which will allow building their own data structures. Moreover, dynamic applications can be built on using the appendable data api.

    The low level apis use the FFI interface for low level API as detailed in the safe_core low level RFC

    APIs for,

    are exposed from the launcher through the REST interface.

    Permission

    Application must request for LOW_LEVEL_API access permission to invoke the low level api to store or read encrypted data. Since low level apis can be used to create data in the network, it might be possible where the data created by the apps cannot be deleted by the user again to retrieve the lost space. Thus, it makes it important to request user for the permission to access low level apis.

    Unauthorised access

    Unauthorised access is granted for reading public data using low level api by default.

    Handle Id

    The low level apis return Handle-Id corresponding to the data type that is being worked with. For example, the structured data will return a DataIdentifier-handle-Id using this Handle-Id, the operations can be performed on the structured data.

    It becomes the applications responsibility to drop the handle after the usage.

    Limitation

    Ability to work with versioned data is exposed only from low level APIs. Launcher's NFS API doesn't support versioning yet.

    Drawbacks

    If the handle id is not cleaned up properly it can lead to memory leak. safe_core can purge if the number of handles goes above a certain threshold.

    Alternatives

    Nil

    Unresolved questions

    Nil



  • MaidSafe Dev Update – October 11, 2016

    Last week we released the second dev tutorial15. Since we are trying to make a new release roughly every two weeks, you can expect Dev Updates between releases to be a bit more quiet

    In addition to working on the next dev tutorial, the front-end team is discussing improvements related to app authorization, permissions and data sharing between applications.

    We are also thinking about how SAFE Launcher can be scaled to various platforms (e.g. mobile, embedded, etc.). For example, SAFE Launcher currently needs to run in the background and continuously proxy HTTP requests to the SAFE Network, but on some mobile platforms that wouldn't necessarily be possible. It would be preferable if mobile apps could communicate directly with the SAFE Network.

    There are multiple approaches we are looking at, and we're planning to have a detailed discussion about potential changes to SAFE Launcher this week. We'll write a summary of the discussion and post it on the Dev Forum. The ideas we'll present will still be open to debate by the community and we’d love to hear your thoughts and feedback once we post that topic

    SAFE Launcher

    APIs that were integrated late into safe_core are being updated in launcher. The API for handling versioned structured data is being updated and tested from launcher.

    The test module for the APIs is being updated to work with electron-mocha6 and once this switch is done, the CI system should be able to run the test suite again.

    As said in the introduction, this week has been more related to discussions on how the launcher should be shaped for the future and how the APIs should be exposed.

    SAFE Core

    We have started the async overhaul this week and are using (relatively much more recent) rust-async-paradigm crates tokio-core1 and futures-rs1. That is the direction Rust community seems to be moving and lot of work is going into them. We have created a separate dev branch to allow master to continue to be usable with present Launcher etc., as it is a big and a breaking change. A lot of work has gone into that already in 2 days and there is steady progress.

    Routing & Vault

    This week we have another member joining the Routing team: Diggory Hardy18

    With 2 new members in Routing and Andreas getting back, we are now looking at our next milestone: user running vaults from home. This is our number one priority.

    The big issue we need to address for vaults from home is being able to prune out vaults that are so low resourced they kill the network. Along with RFC 37 – Disjoint Groups2 (which is already underway), to address the issue of low-powered nodes, we are looking at RFC 45 – Node Ageing9.

    We are also discussing what other changes we need to make before being able to start testing vaults from home again. For example, data chains (which will provide data republish) is not a priority for now, this will come after we are able to test vaults from home.

    Dev Outreach

    With @lightyear starting full time in October, we've started planning our dev outreach strategy. We are currently researching events and have submitted applications to the ones we think most suitable. In comparison with previous years, we are going to be attending more events than ever through 2017 as this is a key part of our strategy in pushing engagement on the back of the alpha and early tutorial releases. We will keep you posted about the events we’ll take part in, of course.

    Each event varies and we will use a combination of talks and workshops where possible. While talks are great for introducing concepts, we are excited by the workshop format, enabling Ben and other members of the team to bring app developers up the learning curve.

    We see dev outreach as a key part of our strategy moving forward and growing the developer outreach team during 2017 will be one of our priorities.

    In addition to attending conferences, we will also be seeding and working with members of the community interested in presenting SAFE at meetups, or run their own developer events and workshops. We will support this activity as much as possible. If you are interested in doing any of these, please get in touch with @lightyear or me (@frabrunelle).

    If you have any suggestions for conferences you think would help us spread the word in the developer community, please post them here1.

    Bitcoin Wednesday, Amsterdam

    We are going to be speaking at Bitcoin Wednesday in Amsterdam on the 7th of December so if you guys can make it through for that event we would love to see you there. We will keep you posted about the details.

    Chaos Communication Congress, Hamburg

    Ben is also planning to be at the Chaos Communication Congress this year2 (December 27th – 30th, 2016 in Hamburg). We are still waiting to hear back whether he got accepted to give a talk there, but he is planning on going either way. And he was wondering who else was planning on coming and whether anyone would be interested in organising an Assembly with him4.

    SAFE Browser

    Last week we created a new Dev Forum topic6 to give feedback on SAFE Browser. @joshuef also created a topic with a general todolist of what he’s working on: Safe Browser Dev Updates (@v0.3.2)2.

    There's an updated release of the Safe Browser to 0.3.2.

    This features a more robust 'safemode' toggle, including a navbar icon showing that you are on the safe network exclusively. This can be toggled by clicking the icon, or going into the File menu. This will reload all tabs upon toggle, blocking/enabling insecure (non safe) content.

    Funding

    MaidSafe’s funding round20 on BnkToTheFuture ends in about 14 hours. £1,246,869 have been raised through 226 backers so far. Thanks again to everyone who invested, supported and helped out with this round!

    Bug reporting

    If you know which component a bug is associated with, feel free to report it on GitHub in the corresponding repo. If you’re not sure where to open an issue, you can simply create a new topic on this forum and add the #bug tag. @lightyear and I are subscribed to the #bug tag, so we will automatically receive a notification whenever someone adds the #bug tag to any topic. We’ll take care of opening an issue on GitHub if necessary.

    If an issue is specifically related to the SAFE API, then it makes sense to use the SAFE Dev Forum. But for issues related to end-user applications (e.g. SAFE Launcher, SAFE Demo App, SAFE Browser, etc.), it's fine to use the SAFE Network Forum. Most users already have an account here

    Also, to make the process of opening an issue on GitHub easier, we will start adding Issue Templates3 to most of our repos.



  • Top 8 Reasons to Invest in Maidsafe Shares

    1) ALL SAFE APPS THAT DO NOT ADOPT THE GPLv3 LICENSE OR USE THE SAFE APIs MUST PAY MAIDSAFE 1% OF THEIR REVENUE ALWAYS. I can't stress how huge this is. Once SAFE's network-effect takes hold, giant companies will be forking over crazy amounts to keep their source code private.

    2) MaidSafe is positioned to receive a large part of the 5% of ALL generated SafeCoins, that are set aside for core devs, for the forseeable futur (until other groups and programmers start helping in this area). That is a huge income. (They may also be among the largest farming groups early on when farming is most profitable)

    3) MaidSafe is currently among--if not The--top SAFE application dev teams in the world, as they created the world's first SAFE email system within a week or two. They could dominate the app scene with ease--especially early on, when SafeCoin sees its giant rises--if they so choose. Don't forget 10% of ALL generated SafeCoins go straight to the app devs.

    4) These stocks are LIMITED, AND MORE MAY NOT BE OFFERED AGAIN! They allow the possibility for future MaidSafe IPO with huge gains, or possible MaidSafe acquisition for giant gains from current tiny price point (of barely $1M USD), or large possible fiat/ SafeCoin dividends paid regularly.

    5) MaidSafe is a hyper-efficient company, as their systems scale infinitely, disrupt companies with billions in profits (and billions in equipment expenses), but has no infrastructure costs of its own. That sets the stage for insane profitability.

    6) Even just the current Alpha release is enough to be licensed to large data centers for profit, as it would eliminate data losses during server crashes. So I know we all aren't happy it isn't 100% finished being fully decentralized yet, but don't overlook the technical marvel of what a networking achievement it is, with its vaults and routing.

    7) Finally, last place is the original USB sticks straight from MaidSafe, and live SafeCoins sent straight to you at launch, both of which could become extremely valuable. Also an amount of free BTC every day from BNK (so the $1k is offset by all this).

    8) oh &..............Con$$$ulting. Period.

    So anyone who believes that SAFE will be successful at all would be foolish to not invest anything into the company that is in the best position to profit hugely from it!!

    (not legal advice etc) (also I made this shortened version of @fergish 's podcast to focus on the parts that talk about profitability. It's the one I keep referencing above.)

    Register & Invest!



  • Safe Browser Dev Updates (@v0.3.6-2)

    0.3.5

    Download:
    https://github.com/joshuef/beaker/releases/tag/v0.3.5

    New pre-release is available here: https://github.com/joshuef/beaker/releases/tag/v0.3.6-0 with updated error handling and more promisey base APIs

    Todo

    Here's the general todolist, vaguely in the order in which I'll attack them:

    SAFE Mode Toggle button/indicator. So you know you're safe.
    Straighten out Appveyor deployfor naming and to only have one link.
    - Linux/Windows safe: protocol integration, so safe: links will (optionally) load the browser
    Investigate localhost development... what can we do here to facilitate this
    - Add launcher detection (helper in safe-js), and appropriate notification
    - Safe network sync for browser data.
    - Add 'You can access the resternet' type screen when in SAFE mode. Instead of ugly broken screen.
    - Setup some helpful default 'favourites' for safe (suggestions welcome, I'd probably start with email [as per @polpolrene's suggestion] and a site list )
    - Investigate beaker shortcut issues.
    - (almost ) update package naming to match maidsafe standards
    - rename windows unpack folder
    - ensure production environment used for builds & remove BeakerDev menu

    safejs specifics

    improve safejs promise error rejection for DNS/NFS/Auth
    - improve safejs promise error rejection for new APIs
    - improve safejs test coverage
    improve safejs response parsing
    improve safejs handling of streaming APIs and html5 streaming




  • Maidsafe-EASIEST API TUTORIAL website:




    It's this new SAFE API Tutorial Website

    Logical extension from APP ZERO. I want it to be extremely approachable, and eventually have tutorials and walkthrough code for anyone creating web apps with safe-js. (the site will ultimately be SAFE-hosted, once the examples work, so that people can click the boxes to make the commands actually execute with launcher)

    Auth was covered by APP ZERO, but it looks like creating the basic folders is the logical next step, and a bit confused by the example code both on the MaidSafe website (bare-js? didn't work) and safe-js gh readme.

    tl;dr NEED HELP WITH CREATING DIRECTORY IN SAFE-JS so I'll tag @joshuef @DavidMtl (did you get your SimpleSafe working?) @happybeing and anyone else can help, submit PR's etc (site repo, & code repo)



  • New authentication flow + Granular permission control

    Today, we are presenting the RFCs related to the new authentication flow that was first explained in the topic about the future of SAFE Launcher.

    New authentication flow

    The current authentication and permissions handling flow relies on SAFE Launcher, a stateful intermediate between any particular app and the network. While this allows for fine-grained control over the access flow and user setup, it has the drawback of requiring the process to always run, keep a local state and proxy all network requests. This is particularly cumbersome on embedded devices and mobile, where we cannot provide this pattern reliably.

    RFC 46 – New Auth Flow proposes to replace SAFE Launcher with SAFE Authenticator (a stateless alternative).

    This RFC outlines a new process to give applications authorised access to act on the SAFE Network with the user's credentials. In particular, it is designed for the mobile and embedded use case in mind.

    Highlights

    At the core stands a rethinking of the data flow and authentication responsibility. Rather than requiring every app to go through the launcher to use the user's credentials, in this approach, the user creates "sub-credentials" per app and assigns them to the app. The app can then access the network with these credentials independently from any other process running. We call these credentials appKeys (as they are a public-private-key-pair attached to a user's maidsafe keys), which an app can interact with the network directly with.

    Apps will have to directly interact with the network. We will restructure safe_core to provide all needed facility for that, as well as the proposed authentication flow against the authenticator. That means that each App has to bundle their own safe_core and link to that library. The library will expose all necessary functionality also through FFI to allow for binding in other environments than rust.

    The root container is the main entry point for the user and most apps to interact with. It is a locally encrypted container stored at a random location on the network known only to the user, that generally only the authenticator has write access to. It reference will be stored in the user’s session packet on account creation. Keys starting with an underscore (_) are reserved for internal usage by the authenticator, while the authenticator may also allow the creation of other keys later.

    When creating a user account the authenticator will create the following minimal set of containers within the root container, each one with its own random network address:
    • _apps/net.maidsafe.authenticator/
    • _documents
    • _downloads
    • _music
    • _pictures
    • _videos
    • _public
    • _publicNames

    It is recommended that apps should only request the rights read and insert - which we will refer to as the BASIC-access. To promote the usage of this, APIs should provide a short-hand for this. Furthermore, the Authenticator may require a multi-step authorization from the user if it encounters an app that asks for more permissions than these and may allow the user to disable specific permissions asked for.

    Discussion topic for RFC 46 – New Auth Flow1


    Granular permission control

    Existing data types don't support granular access control which is required for mobile platforms where access to the network through an intermediary like SAFE Launcher is complicated: each app would have to receive full permissions to modify and fetch data on a user's behalf and private keys to sign it. This compromises security and allows malicious apps to get full access to the user's data.
    To mitigate that, MutableData provides fine-grained access control and more narrow-scoped data operations. This data type is an evolved version of AppendableData. It combines StructuredData and AppendableData features and improves the network efficiency, saves bandwidth, and provides a standardised approach for apps to fetch and mutate data in the network.

    RFC 47 – MutableData proposes to combine StructuredData and AppendableData into a new single data type, MutableData, similar to HashMap.

    <code>pub <span class="hljs-keyword">struct</span> MutableData {
        <span class="hljs-comment">/// Network address</span>
        name: XorName,
        <span class="hljs-comment">/// Type tag</span>
        tag: u64,
        <span class="hljs-comment">// ---- owner and vault access only ----</span>
        <span class="hljs-comment">/// Maps an arbitrary key to a (version, data) tuple value</span>
        data: BTreeMap<Vec<u8>, Value>,
        <span class="hljs-comment">/// Maps an application key to a list of allowed or forbidden actions</span>
        permissions: BTreeMap<User, BTreeSet<Permission>>,
        <span class="hljs-comment">/// Version should be increased for every change in MutableData fields</span>
        <span class="hljs-comment">/// except for data</span>
        version: u64,
        <span class="hljs-comment">/// Contains a set of owners which are allowed to mutate permissions.</span>
        <span class="hljs-comment">/// Currently limited to one owner to disallow multisig.</span>
        owners: BTreeSet<sign::PublicKey>,
    }</code>

    The current features of Appendable Data will be covered by Mutable Data, but rather than just append, Mutable Data will support many discrete actions (e.g. “List entries”, “List keys”, "List values”, “Get value”, “Insert”, “Update”, “Delete”, etc.). These discrete actions will mutate specific key-value pairs directly on the network, rather than having to replace the entire content. Consistency and permissions will be enforced by the vaults. Mutable Data will also allow to create multiple type tags (just like Structured Data).

    The reason for this change is that we want to provide granular permission control over data changes within the network (rather than on the client side). Many things, that so far happened in the launcher (so we can develop faster) but should be done by the network (like stronger enforcement of permissions), will finally move into vaults.

    Secondly, we propose to introduce a new paradigm on the data type itself to handle permissions. Expanding from the owner-or-public-model, we propose to have field on the data type (called “permissions”) which maps signing keys to a list of actions that a key is allowed to perform on a particular data object. Every discrete action is checked by the vaults against the permissions assigned to that key and will only succeed when the required permissions are present.

    Highlights

    The proposed data type defines discrete actions on data that doesn't require a user to replace or fetch the entire structure contents. This is enabled by functionality similar to this of the HashMap data structure or a key-value database such as Redis.

    Permissions can be modified granularly by issuing actions similar to those used to update data. Only owners and apps having the ManagePermission permission must be allowed to issue such requests.

    Pending writes must support granular changes in data or permissions. Ownership transfer and encrypt key changes also must be represented by separate pending writes.

    Limits

    The MutableData data type imposes the following limits:

    • Maximum size for a serialised MutableData structure MUST be 1 MiB;
    • Maximum entries per MutableData MUST be 100;
    • Not more than 5 simultaneous mutation requests are allowed for different keys;
    • Only 1 mutation request is allowed for a single key.

    These numbers were picked as a starting point. They will be profiled and updated via testnets as the performance of the network improves.

    Discussion topic for RFC 47 – MutableData1


    Authorize apps on behalf of the owner to mutate data

    If the requests made by the permitted apps need to be charged these must all go through the Maid-Managers and they should know about the owner that an app is associated with so that they can charge the correct account.

    RFC 48 – Authorise apps on behalf of the owner to mutate data details how apps route their mutation requests to Data-Managers and how revocation works.

    Highlights

    We require all mutations to go via the owner's Maid-Managers. Maid-Managers will keep a list of sign::PublicKey which is the representation of the apps the owner has allowed to mutate the network on their behalf. Maid-Managers will then forward the request to the appropriate Data-Managers. The type MutableData thus does not need to store any signatures and Data-Managers will use the group authority to trust the requester of the mutation. Then they will use the permissions field to allow or disallow the mutation itself.

    Ownership transfers are a special case. The apps are not allowed to do this operation. They will need to ask the Authenticator to do this. Maid-Managers shall enforce that change_mdata_owner only comes from the owner. Data-Managers shall enforce that current owner of the data matches the incoming Maid-Manager group.

    Once the owner decides to revoke permission of an app to mutate network on their behalf, they will go to the Maid-Managers and delete sign_pk for the app. Hence the app can no longer perform any mutating operations on behalf of the owner now as the owner's Maid-Managers will reject it not finding it in the list.

    Discussion topic for RFC 48 – Authorise apps on behalf of the owner to mutate data



  • MaidSafe Dev Update - February 2, 2017 - Test 12

    This will enable some users to run vaults from home. This test doesn't contain the new client API (with mutable data). It's purely for testing the features related to the routing
    update (mainly disjoint sections and resource proof). With such a long
    period between testnets and a considerable upgrade to the codebase, this
    test is essential for us to confirm that the network is improving and
    that we are on track for the remaining features, which required this
    change. The next target for us in terms of features is data republish
    and increased security (node ageing and data chains are large parts of
    the current in house discussions).




    As part of the resource proof paradigm, a spot check for bandwidth is carried out when you start a node. This may extend to CPU at a later stage (this is already included as part of the resource_proof4 library). Your vault will connect to the members of the network's section that it is trying to join, and will try to upload some verifiable and unique data to each of them within five minutes. If successful, the section will accept your node and it will start routing and handling requests.

    Note that each section will only profile one node at a time, so if many users are starting vaults at the same time, many of them will need to retry multiple times until they are accepted as a candidate and start the bandwidth check.

    With the Node Ageing RFC4, the bandwidth parameters will adjust in real-time based on the network requirements. At the moment, we are using "magic numbers" that we have selected through testing to do these spot checks. We have made these joining node resource checks extremely aggressive to begin with. A large percentage of the community vaults (maybe 50%) may not be able to join Test 12. This affects only the vaults though, clients will not be affected by these spot checks.

    Right now, the upload bandwidth requirement is ~6 Mbps. In the long run, we totally intend to make use of every node we can (e.g. for particular specialized tasks). This is just step 1 of the process.

    This test represents a huge refactor and we are very keen to confirm that there are no regression or surprises as we move forward towards Alpha 2.

    SAFE Vault v0.13.0

    You can run only 1 Vault node per LAN – running more than 1 will result in this message:

    More than 1 routing node found on LAN. Currently this is not supported

    Download

    Changelog

    • Migrate to routing 0.28.0.
    • Use a single event loop for routing and safe_vault.
    • Fix issues with account creation and data requests.

    SAFE Launcher v0.10.1

    o connect to Test 12, you need to use SAFE Launcher v0.10.1.

    Each client account is limited to 500 PUTs. A PUT is a request to store a single chunk on the network (a file may contain many chunks). The maximum chunk size is 1 MB.

    Please be aware that we might need to restart this test network, wiping all data that is stored on it. We will announce this in a Dev Update (when possible) prior to taking it down.

    Download

    Documentation

    Changelog

    • The release binaries uses safe_core v0.22.2 to connect with the Test-12.
    • Config file updated to connect with Test-12

    SAFE Demo App v0.6.2


    safe-demo-app.png2244x1624 194 KB

    The latest version of SAFE Demo App (v0.6.2) should continue to work fine.

    When uploading files using SAFE Demo App, the maximum file size is 25 MB per file.

    Download

    Documentation

    SAFE Browser v0.4.3


    safe-browser.png3824x2310 238 KB

    The latest version of SAFE Browser (v0.4.3) should continue to work fine.

    Download

    Other apps

    Apps that were working with TEST 111 should continue to work fine with TEST 12.

    Support

    If you need help with anything related to SAFE Vault, SAFE Launcher or SAFE Demo App, please use the #support category of this forum.

    Where should I report issues?

    If you know which component a bug is associated with, feel free to report it on GitHub in the corresponding repo.

    SAFE Authenticator & API

    Team leader: Krishna

    We made good progress this week resolving the issues we had with the FFI integration. The authenticator plugin1 is now integrated with the safe_browser3 and tested. The initial integration looks good and the tests we have carried have exposed a few bugs and corner cases to be handled.

    Also, the Node.js API (for app developers) has seen some considerable progress. We now have the majority of the APIs tested manually. The File-related APIs have some integration issues and we are looking into those at the moment. Along with fixing the known issues, we are also expanding the test suite and examples for the Node.js API.

    You can take a look at the safe_app_nodejs5 repository to get a better understanding of how the Node.js API can be used. You can only try it on Linux and macOS at the moment. For Windows we need to finalise the build process. We are hoping to continue the steady progress in the upcoming week and announce a stable API for the next update. It's worth mentioning also that we started making plans for the documentation. For the Node.js API, we will likely use documentation.js2 (which uses the JSDoc syntax2).

    SAFE Client Libs & Crust

    Team leader: Spandan

    Carl completed the Mio2 task to provide and test iOS and Android compatibility. We are looking to update Crust to the latest version of Mio. We are currently debugging some bugs/changes in behavior with the latest version of Mio on Windows. We think the bugs might be in Mio, but this is not confirmed yet. Once this is sorted out, we should be good to merge Crust's port to the latest Mio. Carl is again helping out with those bugs.

    Right now, Crust only supports TCP. We are going to work with Carl to support other protocols (e.g. uTP) and improve hole punching as well as some security features we require for "man in the middle" attack avoidance.

    The front-end integration with safe_client_libs is the current workload of the rest of our team and this is being done in parallel with Krishna's team to complete any remaining issues with the new APIs for developers.

    Routing & Vault

    Team leader: Andreas1

    We fixed the remaining issues for the bandwidth resource proof and the feature is now ready for a first test with the community.

    We are also having lots of discussions and meetings about our medium/long-term plans, both technical and organisational.

    Team leaders

    The team leaders (Krishna, Spandan and Andreas) are currently in Troon, Scotland for the next 2 weeks. This is the first time that we have the department heads, managers and local dev team all in one place! We're using the opportunity to discuss: the company roadmap, community engagement (of course ), documentation (including the website), as well as some ongoing partnership opportunities, including our workshop with Glasgow University2 which is due to commence in early March. Being the gracious hosts that we are, we will of course show them the local sights, which in Ayrshire boils down to the interior of one of the many local pubs. 

    Developer outreach

    Nikita (@nbaksalyara local DEF CON group's mini-conference4.

    Also, he recently got accepted in the Mozilla Tech Speakers3 training program. He hopes it will give him more experience for presenting at large events. 



  • MaidSafe Dev Update - February 23, 2017

    So this week had a few surprises in store for us, some good and some not so much in terms of Test-12c. Firstly, we completed the routing patches (that we mentioned in the last dev update) on Friday and early this week, we also had the routing soak test run about 1000 iterations successfully. These are tests that use the mock crust feature to simulate different types of network setups and specific cases we'd like to test. This passed all the way through and was great news.

    We then moved on to testing the production vaults in droplets with internal tests. This was where the not so great news occurred. We hit a few issues very early on that we patched, however, we continued to see a few more issues through the week. It's worth noting that each issue that needs to get debugged and patched takes a bit of time since this involves collecting about 1800 log files and spotting the source of the issue, (fun times . The Routing team went at it hard (so hard that we've made Andreas and Diggory sick) to try and get these resolved ASAP. This was further complicated with the new Crust version we were using in the live network, and involved us running two internal tests to confirm Crust wasn't the variable here (it certainly wasn't, so looking good from that side, more good news from Crust coming up).

    Now for some more good news, we've currently had an internal test network running since yesterday afternoon that has gone through about 2100 churn events. That translates to about ~3500 nodes joining and leaving the network. The internal tests also test some particular corner cases such as lots of merges/splits happening in the network at the same time as nodes joining/leaving. As of writing this dev update, that network is retaining data while the invariant is holding, this is certainly great news. All is not completely done yet though. We have one other issue we've spotted that can occur when a candidate (new node going through resource proof check) is dropped by a section after it splits which in an edge case prevents the split section from ever re-connecting to that candidate. The bug meant that the section did not get rid of the candidate properly after the split which prevented them from even re-establishing the connection to that candidate (who by that time might have been promoted to a full node). This is certainly an issue and @qi_ma is patching this today. We've not run into this edge case in the network within the last 24 hours.

    Once we have the patch for this in place, we intend to keep repeating the internal tests until we can confirm a bug-free network before progressing to the most elaborate test of user run vaults (Test-12c) From the results this week we're hoping that we're not far away, but please do take that with a grain of salt as this situation is very fluid as you can imagine.

    With all this happening at one side this week, we've got some really good news to share from some of the other teams, specifically the progress being made in protocol support from Crust for uTP, and various hole punching tests that we've tried in combination (all this will increase the number of people able to join the network and not just rely on TCP). With frontend APIs getting more reliable, @ustulation and team are also starting their work in Vault to support — yep, you guessed it! — mutable data This will enable the test networks to move from the SD/AD data types, introduce the new data type MD which will then facilitate the Authenticator paradigm in place of Launcher, which would get deprecated at that stage. Just to note this isn't a patch job that we expect to be done with in a week, but we're certainly in motion there to get vaults to support this data type. We've also got the frontend guys going hard at their applications and docs which we intend to share more with you this week. Please check the individual module sections for more information in all these aspects.

    As you already know, we have some of the team in Asia which is great, but as a small team it does hurt dev progress a little back home. So this balance is a fine one that we choose to make, even though it does add a lot of time pressure. We feel sure though that getting the SAFE name out and solidifying relationships with our Asian partners will benefit the SAFE Network and all that is stands for in the long run.

    So in summary, it's been a pretty hectic week all around, got a few people ill (looks like they'll survive just fine), got some of us in Asia spreading the SAFE Network concepts, patching some network bugs from internal tests and progressing along with the MD implementation and Crust updates for protocol support and doc updates of frontend APIs.

    SAFE Authenticator & API

    Last week, @happybeing (again we must thank him for his amazing contributions to this project) had hosted the API documentation for safe_app_nodejs for easier access. We have now published the documentation17 for safe_app_nodejs' master branch, which we will keep up-to-date with the latest changes.

    We're currently improving the build process for the Node.js library and also the safe_browser. The library will be downloading the native dependency at run time and won't have to install Rust and build locally to get the native libraries. @lightyeardependency downloader8 which is a postinstall tool for safe_app_nodejs to download the native library for the current architecture and the platform being installed on.

    @[email protected] have been working on packaging the example app4 – which depends on safe_app_nodejs – using Electron Forge5. This is tested on macOS and we are currently setting it up for use on Windows. We hope to have it integrated soon and test the same across all three platforms. Minor feature updates to the example app have been implemented by @shankar.

    Once we have these tested across platforms, it will make the installation of the safe_app_nodejs library seamless. The browser build process will also be simplified with this WIP PR3 getting merged.

    @[email protected] is working on identifying the helper functions which can be moved to the Rust FFI layer so that the integration with FFI will be easier across languages.

    SAFE Client Libs & Crust

    Quite a few good things are planned for Crust and Rust networking in general. Carl1 having solved the issues with the internals of Mio, is planning to work on an async uTP support on top of Mio. We had given him a few pointers to where the protocol was itself, which is implemented in C++ as libutp, and had asked if he could come up with a list of TODOs. He has given us a rough idea of what API he plans to expose in this new crate. We were recently discussing on having a coded multiplexer in the uTP listener logic that functions on just one socket vs spawning multiple sockets per accept. The general consensus is to spawn a new socket per accept instead of using multiplexers. Although multiplexer would have helped us keep number of used descriptors under control, they would have a significant overhead in terms of CPU cycles and could potentially affect performance. Once we have uTP support, we will certainly be much better off in terms of reaching peers. UDP hole punching trumps TCP by a good factor and uTP wraps a UDP socket with reliability and congestion control layers added on top of it.

    @nbaksalyar in the meantime has been writing more test cases for Crust and looking into the challenges of setting up a virtual network within a machine to test external reachability and other parts of code, which necessitates peers on non-loopback addresses thus posing challenge. While there are ways on *NIX platforms, Windows as usual needs more looking into and out of the box support seems bleak there. If he does figure it out, it might prove to be highly useful for tests in Crust because they would be closer to the real world and will help us iron out deficiencies if at all observed.

    @adam started working on the implementation of MutableData in vaults. We already have this supported in safe_clients_libs via mock vault. Based on the churn handling in the vault library which isn't catered for in the mock, we might have to optimise or revisit certain components of this implementation. MaidManagers should be easier to deal with than DataManagers just now, and that's where we have started. Every successful mutation request now would deduct account balance (previously POST was free and went straight to the DataManagers). Also the final cap is based on total number of mutations, not just PUTs as previously was the case. See this post for more info.

    @ustulation had intermittently done some NAT-traversal testing with different kinds of routers including Symmetric NATs and observed some expectedly unfriendly router behavior. NAT traversal is a cat and mouse game in which we need to find out and try and trick through the weird behavior of routers. All the observations until now are documented in detail with workarounds for successfully reaching peers in this repository13. This is meant to be an independent pluggable crate for robust p2p communication. This is a work-in-progress and current design (and other) details (which are aimed to explain all observations and techniques thoroughly) can be found in the crate docs5. Though still in early stages, a somewhat working chat-engine is already written and tested here9.

    Routing & Vault

    As mentioned in the introduction, this week has mainly involved us patching and updating the routing library based on internal tests. Some of the changes that have been applied this week include:

    • #1357 adds a test with an increased level of churn.
    • #13631 makes the log messages a bit more concise.
    • #13702 fixes problems that occur if a candidate (a joining node doing the resource proof) joins a section that is currently merging with another one.
    • #13714 addresses another section merge bug.

    Developer outreach

    This week's Tech Conference in Jakarta was a great success! See this blog post29 by @nicklambert for more info.

    Also, Ben (@lightyearBob Conf Berlin7 tomorrow. His talk is titled "Privacy-first data structures on SAFE4". We hope it's as good as the excellent talk he gave at FOSDEM 201711 



  • MaidSafe Dev Update - April 13, 2017

    Everything that we had as objectives for Alpha 2 has been met with Test 164. But based on what we've seen from Test 16's limitations, these limitations were the main things that were abused in various ways (e.g. nodes bypassing the relocation process, clients spamming the network with GET requests, nodes having a lot of tunnels). As @dirvinethis post13, these are all things that are currently just spot checks and that were put in and highlighted as limitations that would be resolved down the road with features like node ageing and data chains. So features that were meant to be for Alpha 3 are limitations right now, because we haven't implemented them yet, but those are exactly the things that would solve the kind of issues we experienced in Test 16.

    We have discussed the idea of helping set up a community testnet, it's a double-edged sword though. Test 16 did not show any bugs and, barring the attacker's participation, was fine and would be Alpha 2. We are not sure what can be achieved even with a community testnet right now. We know vaults from home work, we know the limitations and we know devs really want to use the new SAFE API with Mutable Data.

    With this in mind, and instead of delaying Alpha 2 and taking features from Alpha 3 and rushing them into implementation now, we're now going to swap the objectives of Alpha 2 and Alpha 3. We have done the donkey work here, mostly, but need to spend a few short weeks at least soak testing and finalising the mutable data integration to a point where we are happy to release the next testnet. We will try to speed through this with half the team while the other half are already on the routing & vault updates we need for Alpha 3. Alpha 2 will also affect the Alpha 3 code but it's pretty clear where and we can easily cope with any conflicts there. However, to ensure we do not knee-jerk this, we will measure the results carefully and with luck we should do this internal testing over the next few weeks and of course update you all as we progress.

    Alpha 2

    • Mutable Data and authenticator pattern (with the new SAFE API)
    • SAFE Browser DOM APIs and Node.js SDK along with example apps, tutorials and documentation

    Alpha 3

    • Vaults from home (more people should be able to run more nodes and on more platforms)
    • Node Ageing
    • Data Chains, Part 1

    This means that the upcoming testnets will focus on Mutable Data. Like Alpha 1, these will be client-only networks (MaidSafe will run all the vaults). We hope these testnets will trigger a lot of discussion and feedback from the community in terms of how to use the new APIs, how to migrate from SAFE Launcher to SAFE Authenticator, what are things that might need to be addressed or done better, documentation, supporting different platforms, etc. It should be an exciting period!

    Alpha 3 is going to have everything that the vaults had with Test 16 (what we currently have now), plus the missing features without the hacks. We can add temporary hacks here and there, but this is not something we feel is good in the long term of the project and there is not much we would gain out of doing this right now. Instead, we're going to go ahead and implement the actual features that are planned for Alpha 3 in terms of data chains, node ageing2, nodes key as name5, etc. But we're not going to wait for that to be complete before allowing app devs to start using the new SAFE API with Mutable Data, which is why the next testnets will focus on Mutable Data. Mutable Data support will be added to the master branch of routing & safe_vault and, in parallel, we'll implement the features for Alpha 3 in development branches.

    Once the Mutable Data objectives are complete and at a stage where we consider the API stable enough for a "version 1", and if we are not done with node ageing and the first part of data chains (which might be likely), then we will call that the new Alpha 2. Once we are done with the original Alpha 3 objectives (as far as vaults are concerned), we will release Alpha 3 and community members will be able to run vaults from home again. Alpha 2 and 3 may appear very close together in this case.

    Beyond Alpha 3

    • Data Chains, Part 2
    • Data republish and network restarts
    • A security audit of the network
    • Test safecoin
    • Real-time network upgrades
    • Network validated upgrades

    In between alpha networks, there will be many testnets. Each alpha will focus on a few major components. Major features will be added one by one in testnets. Once all those things are in place, there may be other alpha networks, or we may move to beta.

    Upcoming Dev Forum discussions

    We are going to have quite a lot of technical discussions on the SAFE Dev Forum4 over the next few weeks. This will involve discussions from the Mutable Data side and also from the back-end network side (e.g. data chain integration into vaults).

    Here are some of the topics you can expect to see on the Dev Forum in the next few days/weeks:

    • Data Chains (Part 1) design proposal
    • Vault chunk store optimisations (as pointed out by @mavthis topic7, which has lead to @qi_ma100 entries limit8 of Mutable Data
    • Operation recovery for Mutable Data from the network. So a transaction-based process (for some operations initially) to allow rollback in case of failed operations midway (this one is likely going to take a while longer as the guys are still in the early stages of working out the details)

    These will all be open discusions, but at the same time, we will need to consider what is going to work well for the project in the long term and decisions will then need to get made accordingly. To be clear though we will still have ongoing and possibly more blunt debates in house, these are critical to achieve the best outcomes but not always good for public consumption as there would be too many ways that people who would wish to cause harm could take this kind of frank discussion out of context. Any conclusions though would immediately be made public as you would expect and deserve.

    SAFE Authenticator & API

    We finished porting the Markdown editor web application and email application to use the new APIs, but we haven't merged the code to the safe_examples1 repository yet because of an API issue related to Mutable Data. The test cases for Mutable Data are passing, however the API fails when invoked from Electron. We created a CLI sample app to test the Mutable Data APIs and can confirm that the APIs are working fine from the Node.js environment. We are looking into this issue and will have it resolved soon. Once the issue is fixed, we will be able to get the updated examples merged to the safe_examples repository.

    @Kumar tested a few approaches for Java JAR packaging and distribution. He is also trying to get a simple FFI sample tested on Android. JNR6 seems to be a problem on Android. If JNR is not going to work we will switch to JNI4. We would like to have a working sample before starting to implement the actual code for safe_app_java. Ideally, it would be easier if we could use the same source code for both Android and Java desktop apps.

    SAFE Client Libs & Crust

    We are looking into viable solutions to prevent GET and MUTATION spam on the Network. GET spam can be caused by either registered or unregistered accounts, where a (malicious) client makes repeated GET requests of large chunks of Mutable Data (Immutable Data is cached so the benefit to the malicious client is not that huge) and makes the Network do work for them with relatively no effort on their part. It can also be GETs to non-existent data, which will result in failure response but still cause the Network to process such requests. MUTATION spam, while not as obvious as GET spam, can also cause the Network to do a lot of work. In this case, frequent bound-to-fail mutations can be done. As an example, consider a PUT request for a chunk of data (largest allowable size) that already exists on the Network. It will travel to the MaidManagers who will deduct account balance, forward it to the DataManagers who will complain that data exists, MaidManagers will undo the balance deduction on getting failure from DataManagers and will finally relay the failure back to the client. Thus, currently, without spending any balance it is possible to make the Network do a lot of work (thus a spam attack vector). Also, invalid mutations can be done which get rejected at the MaidManagers themselves (e.g. mutations by someone who has run out of account balance or by an unregistered client). Though not as severe as the previous example, they will still cause some level of spam.

    The problem and some of the solutions are being discussed here9. We have some more ideas to eliminate these issues and we'll continue to discuss the merits of each before putting out a proposal, but it seems a pretty solvable issue.

    Then there is the cap of 100 entries for MutableData entries. In the current paradigm we do not actually delete entries but merely blank out the value of the key-value part, and this still occupies space and counts towards the capacity limit (as it needs to be refreshed in churn etc. and treated just like other data). We are also thinking on how to solve this and once we have a proposal we will share it with you.

    Apart from that, there were some bug fixes (e.g. this one2) in safe_client_libs in the dev branch as the front-end team continues to test against it.

    Routing & Vault

    We added section versions to routing that increment whenever a section splits or merges. This simplifies some message flows and already allowed us to get rid of some types of messages, just because it makes it trivial to determine whether the information in a message is more current than the one we already have, or whether it is outdated and can be ignored. It is also a first step towards data chains, which will produce an independently verifiable record of the network's and the data's history.

    In parallel, we have been working on the next step in that direction (Data Chains, Part 1), which will provide a log of all section changes, but not yet data changes. I.e. it will not only version sections, but also produce cryptographic proofs for them, and it will not just include splits and merges, but also additions and removals of nodes.

    We are finally taking the last few steps towards deterministic tests2.

    @qi_mathis topic5, we can hopefully analyse the findings and pick a suitable solution to move this forward.

    Finally, in some other news, Diggory (@dhardy) is off from next week. We'd like to thank him for the time he spent at MaidSafe, he's been of great help in the Routing Design team and in the production cycles as well. We are close to closing some additional positions in the Engineering teams so again there will be more new faces to get to know 



Looks like your connection to Cryptocentral was lost, please wait while we try to reconnect.