What is Common Data Service (CDS) and why it is important for you if you use Power Apps or Power BI?

What is Common Data Service?

Common Data Service, abbreviated as CDS, is a data storage service. Like a database. You can use CDS to store data in the form of tables, which is called as Entities. Common Data Service is a service that is used mainly in the Power Apps portal, however, it is accessible through other Power Platform services and Microsoft Dynamics. the data can be loaded into CDS entities through multiple ways, and it can be also extracted from there through different methods. So you can say it this way that; CDS is a data storage and retrieval system, like a database.

Common Data Service (CDS) is a data storage system, like a database.

CDS includes a set of base entities (tables), but you can add custom entities to it. You can access CDS through other Power Platform services (Power BI, Power Apps, Power Automate…) and some other Microsoft services.

Why CDS is important for you if you are using Power Apps?

If you are using Power Apps, then it means you are creating a mobile application. The mobile application is most likely to work with data, capture information from the user through a data entry form or something and it needs to store the data somewhere. You would need to have a database system that you can store your data and retrieve it.

CDS is free storage for you in the Power Apps environment because you are already paying for the Power Apps license, then you can use CDS for free.

Of course, you can go and build your database in other systems, such as Azure SQL database, but then you need to pay for that service separately, or you might prefer to keep it on-prem in a SQL database, which then you would need to set up a gateway to use it. The choice of which database to use for your Power Apps app is up to you. However, CDS will give you a free, and easy-to-use database system to work with, and build your apps using that.

CDS is the free database service that you can use in Power Apps to store and retrieve the data of your apps.

So in a nutshell; CDS stores your Power Apps data at no extra cost, it is easy to manage. You don’t need a database developer to go and build a database for you to load your data into it. It is a data storage system that can be used by a citizen application developer.

You don’t need to know about databases, or be a database developer, to use CDS. It is built for the citizen app developer.

What the CDS database management system looks like?

Like many other database systems, CDS also has a management tool, which you can go and see entities, edit them and manage them. At the moment, you can use Power Apps portal for this management portal, and you will find CDS under the Power Apps portal like below;


What is the point of CDS if you are using Power BI?

There are two aspects of using CDS if you use Power BI. One is to use CDS as a data source system.

In the world of Power BI, we don’t store the data. We do, however, get data from a data storage system to analyze it. And that is why most of the people in the Power BI world, might not be familiar with CDS, because, from their point of view, this is just another database system, just another data source to get data from.

CDS is another data source that you can use when you Get Data in Power BI.

Another aspect of using CDS in Power BI is to use it as intermediate storage. Intermediate storage for your Power Query transformations. I have explained previously that why you might need to decouple your Power Query transformation layer into Power BI dataflows. Using CDS, you can store the output of dataflows into CDS, like a database, or let’s say, like a data warehouse, and use it for further analysis.

CDS can be your data warehouse if you use dataflows.

I highly recommend you to reach the article I wrote about decoupling the data transformation layer, data modelling layer, and visualization layer in Power BI implementation, which explains how dataflows can be an essential part of implementation for multi-developer architecture;

And the concept of dataflow is nowadays, not just for Power BI, but also for Power Apps. It is Power Platform Dataflows;

What is the storage engine behind the scene for CDS?

CDS stores the data, retrieve it and controls it using Azure services. There are a number of Azure services that are involved with this. Azure SQL DB, SQL elastic pools used for relational data, Blob storage for the non-relational data, and CosmosDB for logs. the screenshot below from Ryan Jones session at Microsoft Ignite 2019 explains how things placed together:

Ways to Load data into CDS

Because CDS is a storage system, you might ask how you can store data into it? what are ways? here is the answer:

  • Power Apps app. You can build an app using Power Apps that store the data into entities of CDS.
  • Power Apps portal using Get Data and Power Query experience.
  • Dataflow: In Power Apps portal, you can create a dataflow and schedule it to load data into CDS.
  • Other services

Way to retrieve data from CDS

You can extract data from CDS in many different ways, including:

  • Power Apps app; You can have forms in your app that show the existing data from entities of CDS.
  • Export data from Power Apps portal
  • Dataflow
  • Power BI, Get data from Common Data Services
  • Other services

How much does it cost for you?

If you are using Power Apps, then you have a license that covers also CDS, so you don’t need to pay anything extra. However, different licenses have different limitations. here you can find out more about it.

If you just have Power BI license, then at the moment of writing this article, the only way for you to use CDS, is to pay for Power Apps licenses. However, remember, for getting data from CDS, you don’t need the license (because someone already created the CDS and is paying for it). You would need a license if you are writing into CDS (through dataflows maybe, which needs its own blog article, which I’ll explain later in another post).


Common Data Service (CDS) is a database system. this database system stores the data in Azure data lake storage (cloud), and you can work with it through a management portal. CDS is a free database if you are using Power Apps licenses, You can then analyze the data of CDS using Power BI. CDS can be also used as a data warehouse layer using dataflows. In other blog articles, I’ll explain more about that scenario.

Reza Rad
Reza Rad is a Microsoft Regional Director, an Author, Trainer, Speaker and Consultant. He has a BSc in Computer engineering; he has more than 20 years’ experience in data analysis, BI, databases, programming, and development mostly on Microsoft technologies. He is a Microsoft Data Platform MVP for nine continuous years (from 2011 till now) for his dedication in Microsoft BI. Reza is an active blogger and co-founder of RADACAD. Reza is also co-founder and co-organizer of Difinity conference in New Zealand.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.
He wrote some books on MS SQL BI and also is writing some others, He was also an active member on online technical forums such as MSDN and Experts-Exchange, and was a moderator of MSDN SQL Server forums, and is an MCP, MCSE, and MCITP of BI. He is the leader of the New Zealand Business Intelligence users group. He is also the author of very popular book Power BI from Rookie to Rock Star, which is free with more than 1700 pages of content and the Power BI Pro Architecture published by Apress.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.

What is L1, L2, and L3 Support Engineering?

Harshana Madusanka Jayamaha

In this article, I’m going to explain about the Software support engineering role with my experience. I was a Level 2 and 3 support Engineer during my career.

L1 — Level 1
L2 — Level 2
L3 — Level 3
Ticket — Incident

L1 support includes interacting with customers, understand their issue and create tickets against it. The ticket then routed to the relevant L2 support ( Integration support, Server & Storage support, etc …). L1 support Engineers have basic knowledge of product/service and skill to troubleshoot a very basic issue like password reset, software installation/uninstallation/reinstallation.

L2 support manages the tickets which routed to them by L1( L2 support also can create tickets against any issue noticed by them). They have more knowledge, more experience in solving related complex issues and can guide/help L1 support folks job in troubleshooting. If the solution not provided at this level then escalate to the L3.

L3 is the last line of support and usually comprise of a development team which addresses the technical issues. They are expert in their domain and handle the most difficult problems. They do the code changes, research and develop the solution for challenging new or unknown issues.

What is Power Apps?

Power Apps is a suite of apps, services, connectors and data platform that provides a rapid application development environment to build custom apps for your business needs. Using Power Apps, you can quickly build custom business apps that connect to your business data stored either in the underlying data platform (Microsoft Dataverseor in various online and on-premises data sources (SharePoint, Microsoft 365, Dynamics 365, SQL Server, and so on).

Power Apps.

Apps built using Power Apps provide rich business logic and workflow capabilities to transform your manual business processes to digital, automated processes. Further, apps built using Power Apps have a responsive design, and can run seamlessly in browser or on mobile devices (phone or tablet). Power Apps "democratizes" the custom business app building experience by enabling users to build feature-rich, custom business apps without writing code.

Power Apps also provides an extensible platform that lets pro developers programmatically interact with data and metadata, apply business logic, create custom connectors, and integrate with external data.

For more information:

Power Apps for app makers/creators

Using Power Apps, you can create three types of apps: canvasmodel-driven, and portal. More information: Overview of creating apps in Power Apps.

To create an app, you start with make.powerapps.com.

  • Power Apps Studio is the app designer used for building canvas apps. The app designer makes creating apps feel more like building a slide deck in Microsoft PowerPoint. More information: Generate an app from data

  • App designer for model-driven apps lets you define the sitemap and add components to build a model-driven app. More information: Design model-driven apps using app designer

  • Power Apps portals Studio is a WYSIWYG design tool to add and configure webpages, components, forms, and lists. More information: Power Apps portals Studio anatomy

Ready to convert your ideas into an app? Start here: Planning a Power Apps project

Power Apps for app users

You can run apps that you created, or that someone else created and shared with you, in browser or on mobile devices (phone or tablet). More information:

Power Apps for admins

Power Apps admins can use Power Platform admin center (admin.powerplatform.microsoft.com) to create and manage environments, get real-time, self-help recommendations and support for Power Apps and Power Automate, and view Dataverse analytics. More information: Administer Power Platform

Power Apps for developers

Developers are app makers who can write code to extend business app creation and customization. Developers can use code to create data and metadata, apply server-side logic using Azure functions, plug-ins, and workflow extensions, apply client-side logic using JavaScript, integrate with external data using virtual entities and webhooks, build custom connectors, and embed apps into your website experiences to create integrated solutions. More information:

Power Apps and Dynamics 365

Dynamics 365 apps, such as Dynamics 365 Sales, Dynamics 365 Customer Service, Dynamics 365 Marketing also use the underlying data platform (Dataverse) used by Power Apps to store and secure data. This enables you to build apps using Power Apps and Dataverse directly against your core business data already used within Dynamics 365 without the need for integration. More information: Dynamics 365 and Dataverse

Try Power Apps for free

You can build Power Apps for free. Simply sign-in to Power Apps. For more information, go to Sign in to Power Apps for the first time. Initially, you'll have access to the default environment.

A license is needed to play the apps made with Power Apps. You can both build and play Power Apps for free by signing up either for a 30 day trial or developer plan.

Purchase Power Apps

If you have decided to purchase Power Apps, see here for detailed information: Purchase Power Apps.

Power Apps US Government plans

Power Apps US Government consists of several plans for US government organizations to address the unique and evolving requirements of the United States public sector. The Power Apps GCC environment provides compliance with federal requirements for cloud services, including FedRAMP High, DoD DISA IL2, and requirements for criminal justice systems (CJI data types). More information: Power Apps US Government

What is the Power Platform?

Microsoft rapidly innovate, update and release new products and solutions, which can make staying on top of changes difficult. However, this frequent pace of innovation makes Microsoft technologies very exciting. One of the biggest new areas from Microsoft that you will read a lot about is the Power Platform.

Note: This article has been updated as Flow has been re-branded as Power Automate. Some visuals remain branded as Flow.

What is the Power Platform?

The ‘Power Platform’ is a collective term for three Microsoft products: Power BI, PowerApps and Power Automate (previously known as Flow). They provide the means to help people easily manipulate, surface, automate and analyse data and can be used with Office 365 and Dynamics 365 (as well as other third-party apps and other Microsoft services). The Power Platform is possible thanks to the Common Data Service (or CDS), which is essentially the underlying data platform that provides a unified and simplified data schema so that applications and services can inter-operate.

Power Platform Overview

Why is the Power Platform so important?

In this digital age, we are extremely reliant on data – and the amount of data companies are creating is continually increasing. While all this data is inevitable, it is useless unless companies gain insights and meaning from it - to gain tangible value.

Historically, data analysis, app creation or automation would be achieved by IT/Development teams. This would require staff to outline their requirements and aims, submit these requests to their IT department (or even an external partner) and then see whether it was approved and subsequently, wait for it to be built. This would be time-consuming and would use valuable resources internally or be costly if fulfilled externally. What're more, those requesting the solution would tend to have an immediate need and waiting for weeks could cause internal delays.

This is why the Power Platform is so exciting. The Power Platform enables data democratisation – the ability for digital information to be accessible to the typical (non-technical) end user. It provides three technologies that allow staff to do more with their data themselves without coding knowledge. While it doesn’t allow the intricacies and flexibility of custom coding, it does provide a simple method for most users to be able to create, automate or analyse their data in ways which have never been possible for the average worker.

PowerApps, Power Automate & Power BI explained

As mentioned, the Power Platform consists of three technologies - let's look at these three in more detail. 


PowerApps- Logo

PowerApps is a low-code approach to custom app development, allowing users to quickly create apps with a ‘point and click’ approach. It allows you to:

  • Build mobile-friendly apps quickly and without development knowledge and reducing pressure on busy IT teams
  • Connect to and surface data from your business applications, such as Dynamics 365 and Office 365 (and also third-party apps)
  • Surface key data into a user-friendly app to help data entry – meaning users only see the information they need to fulfil a particular task


Power Automate used to be known as Microsoft Flow. This allows you to create automated workflows between your Microsoft services or other third-party applications, which allows staff to avoid carrying out repetitive tasks and save valuable time. It allows you to:

  • Use pre-built automation templates for common automations (within the Flow Gallery)
  • Create you own automations by connecting various applications, such as Outlook, SharePoint Dynamics 365 or non-Microsoft apps like Twitter, Asana, Gmail, MailChimp etc.
  • Set up triggers, alerts, automated emails, push notifications and much more – with no coding and in minutes
  • Overall it allows you to save time, reduce human error and streamline your processes


Power BI is a business analytics tool which allows you to easily connect to data sources, create visuals and gain business intelligence quickly. It allows you to:

  • Click and connect with Microsoft and third-party cloud services, as well as on-premise data sources
  • Easily manipulate data and create visuals, such as charts, dashboards, maps and many more – so you can present your data in an easy-to-digest format
  • Use natural language to query data and get results (i.e. “show me our sales pipeline for 2018 by month, by sales person”)
  • Overall, allowing you to easily analyse and make sense of complex data to enable continual improvement

How does the Power Platform fit with the wider Microsoft strategy?

The Power Platform connects to a wide range of data sources – including third-party apps such as Google Analytics and Twitter, however it is extremely powerful when working with Office 365 and Dynamics 365. Microsoft pitch the Power Platform as a way to “unlock the potential of Dynamics 365 and Office 365 faster than you ever thought possible” so you can easily extend, customise and integrate these services.

The Power Platform is going to be a big area of investment from Microsoft and as it is still fairly new, we expect that it will be regularly updated and improved as the products mature. What’s more, as Microsoft continue to focus on bringing all their technologies closer together, we can expect the Power Platform to be a key player for this – helping connect services like Microsoft 365, Dynamics 365 and Azure easily with a low-code/no-code approach.

How do I get it? Power Platform licensing and pricing

There is not currently a licensing bundle for all three products together. Instead you can purchase the three separately (and can mix and match) - or, if you haven't tried them you can use free versions or free trials to test their capabilities. 

PowerApps and Power Automate licensing is fairly complex. You can read the latest licensing guidance here: https://docs.microsoft.com/en-us/power-platform/admin/powerapps-flow-licensing-faq If you have any questions around licensing and pricing we recommend speaking to a Microsoft Partner. 

Power BI – There are three versions available, Free, Pro and Premium, which you can compare here: Power BI Pricing (Microsoft).

  • Free
  • Pro - note, required for sharing Power BI reports and dashboards
  • Premium (POA)



What’s included in the Microsoft Modern Workplace?

As businesses evolve it is very easy for their business technology and processes to quickly become out of date and potentially result in inefficient operations that predominantly affect customer experience and profit margins.

The Workplace Evolution published by Harvard Business Review found that 78% of senior executives in enterprise businesses believe fostering a modern workplace strategy is essential. However, only 31% think their company is forward-thinking enough to do so.

Changing a fixed mindset into a forward-thinking one within any business is no easy task, and with the implications of remote working due to the pandemic, many businesses have undergone rapid transformation over the past few months by implementing Modern Workplace technology within their businesses. This enables their employees to be supported by the leading technology that enables them to work smarter and be more productive, collaborate with remote teams and reap the other many benefits that Microsoft 365 offers.

What’s included in the Microsoft Modern Workplace?

The Microsoft Modern Workplace is one that operates using the suite of Microsoft 365 technologies and productivity applications that harness the power of the Cloud.

Microsoft Modern Workplace applications improve employee productivity and satisfaction and create more seamless communication across the business whilst promoting collaboration and maintaining the security and integrity of systems and data.


What is a Modern Workplace?

The definition of a Modern Workplace is an operational setup which has been professionally designed to meet both the physical and technological needs of both your business and its employees.

A Modern Workplace drives company-wide business transformation by utilising the latest Microsoft technology to power and streamline business operations and empower employees to do their best work around the clock.

What’s included within the Microsoft Modern Workplace?

Microsoft 365, includes Office 365, Windows 10 Enterprise, and Enterprise Mobility + Security and a variety of productivity and collaboration tools designed to support modern ways of working, help facilitate digital transformation, and most importantly keep your business secure.

How does the Microsoft Modern Workplace facilitate automation?

Within Microsoft 365 is Microsoft Power Automate (formerly Flow) which enables you to implement both Digital and Robotic Process Automation (RPA) across the business. By automating tasks you can quickly boost productivity, giving employees more time to focus on innovation than administrative tasks. You can also automate time-consuming tasks using the built-in AI capabilities and integrate with over 100 applications such as Microsoft Dynamics 365, Twitter, MailChimp, Google Analytics etc.

How does the Microsoft Modern Workplace promote collaboration?

When internal and external teams come together and collaborate to solve a problem the magic happens. Facilitating collaboration can prove tricky especially with remote/shift workers or multi-site offices. However, the collaboration applications included within the Microsoft Modern Workplace such as Microsoft Teams, Microsoft Teams for Education, Office 365 and SharePoint enable employees to easily work together on documents no matter their location or time of day. Windows 10 Virtual Desktop takes this even further by allowing users to access their desktop from any device.

Microsoft Teams in particular is constantly evolving and adding new features to improve accessibility and user experience.

This boosts productivity, can raise morale and enables people within different teams to come together. The more flexible it is for employees to collaborate the more collaboration will occur.

How secure is the Microsoft Modern Workplace?

Within Microsoft 365, the security stack gives us the insights to proactively defend against advanced threats, such as malware, phishing, and zero-day attacks as well as identity, app, data, and device protection with Azure Active Directory, Microsoft Intune, and Windows Information Protection.

What are the reasons to adopt a Modern Workplace?

Here are some common reasons why businesses choose to adopt the Microsoft Modern Workplace over a traditional setup:

Disparate communication channels – employees communicate through a variety of different channels which enables messages to get lost in translation, there is limited visibility and a lack of control.

Stand-alone platforms – business management platforms are implemented that have very limited integration and automation with each other. This makes the risk of error high and increases workload.

Data/Team Silos – Teams work in silos with limited visibility to other parts of the business. Data is stored in numerous locations and access is restricted due to the setup.

Hardware – Desktop PCs are commonly used, not all employees have access to laptops or tablets which disables remote working.

Technology – The technology you have implemented has scalable limitations and does not completely meet the needs of your employees.

Security – There is limited or no device or cyber security management in place putting your business at risk in the event of a breach.


 Ref: https://www.microsoft.com/en-us/itshowcase/microsoft-365 


1C:Enterprise in the cloud

The concept of cloud services for business applications is as simple as moving the application servers from the on-premises network to the Internet. The end users continue working with the same software (either the native client or the web client); the only thing required is an Internet connection. They no longer need to log on to the local enterprise network (directly or through VPN). Moreover, if the enterprise uses the SaaS model, the end users do not need to worry about software administration and updates any longer—the cloud service provider hosting your application servers will manage these tasks.

Eye catcher image: the author of this article illustrates the "1C:Enterprise in the cloud" concept by using simple objects: clouds, banner, aircraft, parachute.

1C:Enterprise applications support both HTTP and HTTPS connections, making for a seamless transition of 1C:Enterprise application servers to the Internet. That's all you need to create a basic 1C:Enterprise cloud solution.
The only difference between this scenario and the on-premises installation is the location of the application server. To provide a service to a new company, one needs at least a new 1C:Enterprise infobase and a physical computer or a virtual machine running 1C:Enterprise. Therefore, application administration costs grow with the number of companies that utilize the service.

Multitenancy and data separation

To reduce application administration costs, you can now have multiple companies work with a single application instance. Of course, the application must be designed for shared work. And the application business logic in the shared work scenario must be identical to the business logic in the scenario where each company uses its own application.

This application architecture type is known as multitenancy. We can explain the multitenancy concept using the following example: a regular application is a house that provides its infrastructure (walls, roof, water system, heating system, etc.) to a family that lives there. A multitenant application is a house with multiple apartments, each apartment having access to the same shared infrastructure that is implemented at the basic house level.

In the simplest terms, the goal of multitenancy is reducing application maintenance costs by pushing the infrastructure maintenance to a higher level. It is similar to reducing costs by selling standard out-of-box applications (that might require some customization) instead of writing applications for specific customers from scratch. The difference is that out-of-box solutions reduce the development cost, while cloud solutions reduce the maintenance cost.

One of the multitenancy aspects is the separation of application data. An application stores data of all companies in a single database, however, each company can only access its own data (such as business documents and employee lists). Some reference data (such as legislation and regulations) can be made available to all companies. 1C:Enterprise applications can use the data separation functionality provided by the 1C:Enterprise platform.

Cloud services

Recently, a group of 1C:Enterprise developers faced the task of developing a cloud service for leasing 1C:Enterprise applications based on SaaS model. Moreover, the service needed to be an out-of-box solution, a "cloud in the box" including everything the customer may need to deploy infrastructure for leasing 1C:Enterprise applications (or individual applications based on 1C:Enterprise).

So what is an ideal cloud service from the end user's point of view? A store with shelves filled with solutions: accounting, reporting, payroll and human resources, and so on. A customer fills their cart and pays at the cash desk (however, they pay a rent instead of a one-time payment). Then the customer decides which of their employees will have access to accounting, payroll, and HR, as well as other solutions.

What is an ideal cloud service from the service provider's point of view? Basically, it's a large store they own. They have to fill the shelves with goods (software), add new goods, and make sure the customers pay promptly. The service must also provide horizontal scalability, access to solution demos (test drive), and centralized user administration tools.

Of course one can implement all this functionality directly in the applications. However, this means duplicating a large amount of code. It is better to optimize the solutions by implementing their common functionality and administration tools in a product that will serve as an entry point for users of cloud services.
This is how 1cFresh technology was developed. 1C customers and partners use it in their SaaS services and private clouds. 1C Company has its own application lease service based on 1cFresh: 1cFresh.com and 1C:AccountingService (both in Russian).

The service functionality is divided between the following major components based on 1C:Enterprise and Java technologies:
  • Service website. A single entry point for all users.
  • Service manager. An administration and coordination tool governing all service components.
  • Application gateway. The component that provides horizontal scalability.
  • Service agent. The component that provides all utility functions, such as application version updates or backup creation.
  • Service forum. A forum for service users and service provider representatives.
  • Availability manager. The "Service temporarily unavailable" board that informs users about service unavailability or the unavailability of service parts, the board itself is available even if central service components have failed.
Simplified 1cFresh component chart (with some components omitted)

Let us review the major service components in detail.

Service website

The site that provides the interface for service users is written in Java. It serves as a store shelf where users can choose applications to rent and try their demos. In addition to that, it is where users register, create application user accounts, read news and browse service online help. The 1cFresh.com page (in Russian) is exactly the "out-of-box" website, without any customizations.

A service can include any number of 1C:Enterprise server clusters that run 1C:Enterprise applications. Each cluster is registered in the service manager. 1C:Enterprise servers can run on both Windows and Linux computers. For example, the service at 1cFresh.com uses both Windows servers (with MS SQL Server DBMS for storing application data) and Linux servers (with PostgreSQL).

The cloud service administrators access the service via the service manager user interface. They use it to add 1C:Enterprise servers and applications, update application versions, manage user accounts, and perform other administrative tasks. Some of the operations, such as application updates, are delegated to the service agent component. The service manager communicates with the service agent via a web service.

Service agent

The service agent is a 1C:Enterprise application. It performs administrative operations on the service infobases, which include application version updates, scheduled backups, and gathering service operation statistics.

Application gateway

The application gateway is written in Java. It is responsible for the horizontal scalability of the service. It redirects service users to appropriate application servers.

Service forum

The service forum is a location where service users and service provider representatives can discuss the service and the applications available in that service. It is written in Java.

Availability manager

Some service features or even the entire service might be temporarily unavailable to end users. For example, an application is usually unavailable to end users while it is being updated, or the entire service might be unavailable during maintenance hours. The availability manager is a 1C:Enterprise application that displays messages about the unavailability of service resources to website and forum users even if all other service components, including the central service manager component, are not available.

1C:Enterprise infobases

1C:Enterprise infobases store application data. New infobases are added to serve as parts of scalability units. Each scalability unit is deployed as a single module and contains the following parts:
  • 1C:Enterprise server cluster
  • DBMS server that stores infobase data
  • One or two web servers (two ensure fault tolerance) that process HTTP requests to infobases belonging to the scalability unit
A scalability unit failure only affects the customers that work with the infobases belonging to the unit.

More service facts
  • The service supports a technology similar to OpenID that allows storing user authentication data in a single database. Therefore, you can set up Single Sign-On for the service and its users will be able to access all their applications (for example, accounting or payroll calculation) and the forum with a single user name and password.
  • Users can transfer local 1C:Enterprise application data (for example, accounting records) to the cloud and back.
  • Users can create standalone workstations (file infobases stored at their local computers). They do not need the Internet or service connections to work with such infobases. At the same time, they can use the service data exchange functionality to synchronize their local data with the cloud.
  • Users can set up automatic data exchange between the applications published in the service (for example, between accounting and payroll calculation applications). This minimizes the efforts required to input data because data entered into one application becomes available for all other applications.
  • A backup creation system is available. A user can initiate backup creation at any time, or schedule daily, monthly, and yearly backups.
  • The 1cFresh technology includes the data delivery feature. The service manager stores reference data that is always up-to-date and provides this data to all applications.
  • A service can run multiple versions of any application. These applications can use multiple 1C:Enterprise platform versions.
  • Users can update the applications that they use to access their infobases.
  • 1cFresh features the following tools for error identification and analysis:
    • Gathering infobase error data.
    • Writing this data to the error log of the service manager infobase.
    • Viewing error details. A service administrator can view the entire error log or filter it by infobase or application.
  • 1cFresh includes the "showcase" feature, which provides the option to run multiple cloud services on a single platform. A showcase is an Internet resource that provides services. From the user’s point of view, a showcase is an independent website with business applications. For example, a single website platform that belongs to a service provider can run several sites located in different domains, one featuring a showcase of small business applications, another with applications for public institutions, the third one with applications for medical institutions, and so on. Also, service providers can advertise each resource as an independent service.
  • The service includes a Feedback center where users can submit their feedback and feature requests. It is implemented as an application module that features a list of user posts and comments to these posts, voting for posts and commenting on them, as well as submitting feedback and feature requests. To include this functionality in an application, an administrator simply enables the subsystem where it is implemented.
  • Subscribers of 1cFresh-based services pay a subscription fee to the service provider. Flexible pricing options are available.
  • The service provides a wide range of options for viewing its usage statistics. One can use the statistics to determine the service load, obtain average key indicators of stable service work (for future evaluation of possible deviations), determine the periods of minimum and maximum service load (for planning scheduled maintenance), and more.
  • The option to gather application business statistics is available. Application developers can use the statistics to improve their understanding of application usage scenarios and to identify bottlenecks.

Applications compatible with cloud services

To be able to run in the cloud, 1C:Enterprise applications must meet the SaaS mode requirements. For the detailed list of requirements, see 1cFresh documentation.

The requirements include the use of data separation functionality, as well as implementations of remote administration functions, data import and export, backup generation, and more. Cloud applications must offer identical behavior between the thin client and the web client, they must not include OS-dependent features (because a cloud server might run either Windows or Linux), lengthy server calls are not recommended, and so on.

Cloud applications can have mobile application clients developed using the mobile 1C:Enterprise platform.

Cloud application customization

Thousands of users from hundreds of companies can work with a single cloud application instance. They might require custom application features. Thus, service providers need tools to customize applications for certain user groups.

1cFresh provides two kinds of application customization tools:
  • External reports and external data processors. These customization tools, well known to 1C:Enterprise application users, were enhanced for cloud operations.
  • Configuration extensions. Extensions are plug-ins that add functionality to applications without changing them. Currently, extensions do not support all of the configuration objects, however, we are working on making this function available.


We believe cloud service development to be a promising trend worthy of significant resource investment.
1cFresh cloud service fully complies with the cloud service definition provided by IDC:

According to the Gartner definition, 1cFresh service type is an Application Platform that operates as a Service (aPaaS): "Application Platform as a Service (aPaaS) is a form of PaaS that provides a platform to support app development, deployment and execution in the cloud." (source)


1C Developer team


Train a deep learning image classification model with ML.NET and TensorFlow

This sample may be downloaded and built directly. However, for a succesful run, you must first unzip assets.zip in the project directory, and copy its subdirectories into the assets directory.

Source Code - Click to download

Understanding the problem

Image classification is a computer vision problem. Image classification takes an image as input and categorizes it into a prescribed class. This sample shows a .NET Core console application that trains a custom deep learning model using transfer learning, a pretrained image classification TensorFlow model and the ML.NET Image Classification API to classify images of concrete surfaces into one of two categories, cracked or uncracked.



The datasets for this tutorial are from Maguire, Marc; Dorafshan, Sattar; and Thomas, Robert J., "SDNET2018: A concrete crack image dataset for machine learning applications" (2018). Browse all Datasets. Paper 48. https://digitalcommons.usu.edu/all_datasets/48

SDNET2018 is an image dataset that contains annotations for cracked and non-cracked concrete structures (bridge decks, walls, and pavement).

The data is organized in three subdirectories:

  • D contains bridge deck images
  • P contains pavement images
  • W contains wall images

Each of these subdirectories contains two additional prefixed subdirectories:

  • C is the prefix used for cracked surfaces
  • U is the prefix used for uncracked surfaces

In this sample, only bridge deck images are used.

Prepare Data

  1. Unzip the assets.zip directory in the project directory.
  2. Copy the subdirectories into the assets directory.
  3. Define the image data schema containing the image path and category the image belongs to. Create a class called ImageData.
class ImageData
    public string ImagePath { get; set; }

    public string Label { get; set; }
  1. Define the input schema by creating the ModelInput class. The only columns/properties used for training and making predictions are the Image and LabelAsKey. The ImagePath and Label columns are there for convenience to access the original file name and text representation of the category it belongs to respectively.
class ModelInput
    public byte[] Image { get; set; }
    public UInt32 LabelAsKey { get; set; }

    public string ImagePath { get; set; }

    public string Label { get; set; }
  1. Define the output schema by creating the ModelOutput class.
class ModelOutput
    public string ImagePath { get; set; }

    public string Label { get; set; }

    public string PredictedLabel { get; set; }

Load the data

  1. Before loading the data, it needs to be formatted into a list of ImageInput objects. To do so, create a data loading utility method LoadImagesFromDirectory.
public static IEnumerable<ImageData> LoadImagesFromDirectory(string folder, bool useFolderNameAsLabel = true)
    var files = Directory.GetFiles(folder, "*",
        searchOption: SearchOption.AllDirectories);

    foreach (var file in files)
        if ((Path.GetExtension(file) != ".jpg") && (Path.GetExtension(file) != ".png"))

        var label = Path.GetFileName(file);

        if (useFolderNameAsLabel)
            label = Directory.GetParent(file).Name;
            for (int index = 0; index < label.Length; index++)
                if (!char.IsLetter(label[index]))
                    label = label.Substring(0, index);

        yield return new ImageData()
            ImagePath = file,
            Label = label
  1. Inside of your application, use the LoadImagesFromDirectory method to load the data.
IEnumerable<ImageData> images = LoadImagesFromDirectory(folder: assetsRelativePath, useFolderNameAsLabel: true);
IDataView imageData = mlContext.Data.LoadFromEnumerable(images);

Preprocess the data

  1. Add variance to the data by shuffling it.
IDataView shuffledData = mlContext.Data.ShuffleRows(imageData);
  1. Machine learning models expect input to be in numerical format. Therefore, some preprocessing needs to be done on the data prior to training. First, the label or value to predict is converted into a numerical value. Then, the images are loaded as a byte[].
var preprocessingPipeline = mlContext.Transforms.Conversion.MapValueToKey(
        inputColumnName: "Label",
        outputColumnName: "LabelAsKey")
        outputColumnName: "Image",
        imageFolder: assetsRelativePath,
        inputColumnName: "ImagePath"));
  1. Fit the data to the preprocessing pipeline.
IDataView preProcessedData = preprocessingPipeline
  1. Create train/validation/test datasets to train and evaluate the model.
TrainTestData trainSplit = mlContext.Data.TrainTestSplit(data: preProcessedData, testFraction: 0.3);
TrainTestData validationTestSplit = mlContext.Data.TrainTestSplit(trainSplit.TestSet);

IDataView trainSet = trainSplit.TrainSet;
IDataView validationSet = validationTestSplit.TrainSet;
IDataView testSet = validationTestSplit.TestSet;

Define the training pipeline

var classifierOptions = new ImageClassificationTrainer.Options()
    FeatureColumnName = "Image",
    LabelColumnName = "LabelAsKey",
    ValidationSet = validationSet,
    Arch = ImageClassificationTrainer.Architecture.ResnetV2101,
    MetricsCallback = (metrics) => Console.WriteLine(metrics),
    TestOnTrainSet = false,
    ReuseTrainSetBottleneckCachedValues = true,
    ReuseValidationSetBottleneckCachedValues = true,

var trainingPipeline = mlContext.MulticlassClassification.Trainers.ImageClassification(classifierOptions)

Train the model

Apply the data to the training pipeline.

ITransformer trainedModel = trainingPipeline.Fit(trainSet);

Use the model

  1. Create a utility method to display predictions.
private static void OutputPrediction(ModelOutput prediction)
    string imageName = Path.GetFileName(prediction.ImagePath);
    Console.WriteLine($"Image: {imageName} | Actual Value: {prediction.Label} | Predicted Value: {prediction.PredictedLabel}");

Classify a single image

  1. Make predictions on the test set using the trained model. Create a utility method called ClassifySingleImage.
public static void ClassifySingleImage(MLContext mlContext, IDataView data, ITransformer trainedModel)
    PredictionEngine<ModelInput, ModelOutput> predictionEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(trainedModel);

    ModelInput image = mlContext.Data.CreateEnumerable<ModelInput>(data,reuseRowObject:true).First();

    ModelOutput prediction = predictionEngine.Predict(image);

    Console.WriteLine("Classifying single image");
  1. Use the ClassifySingleImage inside of your application.
ClassifySingleImage(mlContext, testSet, trainedModel);

Classify multiple images

  1. Make predictions on the test set using the trained model. Create a utility method called ClassifyImages.
public static void ClassifyImages(MLContext mlContext, IDataView data, ITransformer trainedModel)
    IDataView predictionData = trainedModel.Transform(data);

    IEnumerable<ModelOutput> predictions = mlContext.Data.CreateEnumerable<ModelOutput>(predictionData, reuseRowObject: true).Take(10);

    Console.WriteLine("Classifying multiple images");
    foreach (var prediction in predictions)
  1. Use the ClassifyImages inside of your application.
ClassifySingleImage(mlContext, testSet, trainedModel);

Run the application

Run your console app. The output should be similar to that below. You may see warnings or processing messages, but these messages have been removed from the following results for clarity. For brevity, the output has been condensed.

Bottleneck phase

Phase: Bottleneck Computation, Dataset used:      Train, Image Index: 279
Phase: Bottleneck Computation, Dataset used:      Train, Image Index: 280
Phase: Bottleneck Computation, Dataset used: Validation, Image Index:   1
Phase: Bottleneck Computation, Dataset used: Validation, Image Index:   2

Training phase

Phase: Training, Dataset used: Validation, Batch Processed Count:   6, Epoch:  21, Accuracy:  0.6797619
Phase: Training, Dataset used: Validation, Batch Processed Count:   6, Epoch:  22, Accuracy:  0.7642857
Phase: Training, Dataset used: Validation, Batch Processed Count:   6, Epoch:  23, Accuracy:  0.7916667

Classification Output

Classifying single image
Image: 7001-220.jpg | Actual Value: UD | Predicted Value: UD

Classifying multiple images
Image: 7001-220.jpg | Actual Value: UD | Predicted Value: UD
Image: 7001-163.jpg | Actual Value: UD | Predicted Value: UD
Image: 7001-210.jpg | Actual Value: UD | Predicted Value: UD
Image: 7004-125.jpg | Actual Value: CD | Predicted Value: UD
Image: 7001-170.jpg | Actual Value: UD | Predicted Value: UD
Image: 7001-77.jpg | Actual Value: UD | Predicted Value: UD

Improve the model

  • More Data: The more examples a model learns from, the better it performs. Download the full SDNET2018 dataset and use it to train.
  • Augment the data: A common technique to add variety to the data is to augment the data by taking an image and applying different transforms (rotate, flip, shift, crop). This adds more varied examples for the model to learn from.
  • Train for a longer time: The longer you train, the more tuned the model will be. Increasing the number of epochs may improve the performance of your model.
  • Experiment with the hyper-parameters: In addition to the parameters used in this tutorial, other parameters can be tuned to potentially improve performance. Changing the learning rate, which determines the magnitude of updates made to the model after each epoch may improve performance.
  • Use a different model architecture: Depending on what your data looks like, the model that can best learn its features may differ. If you're not satisfied with the performance of your model, try changing the architecture.

Ref: https://docs.microsoft.com/en-us/samples/dotnet/machinelearning-samples/mlnet-image-classification-transfer-learning/ 

ML.NET and Model Builder

ML.NET is an open-source, cross-platform machine learning framework for .NET developers. It enables integrating machine learning into your .NET apps without requiring you to leave the .NET ecosystem or even have a background in ML or data science.

We are excited to announce new versions of ML.NET and Model Builder!

In this post, we’ll cover the following items:

  1. Model Builder Preview
  2. ML.NET v1.5.5
  3. Virtual ML.NET Community Conference
  4. Feedback
  5. Get started and resources

Model Builder Preview

This preview brings a lot of big changes to Model Builder, and we’re excited to get your feedback on all the new features which include:

  • Config-based training with generated code-behind files
  • Restructured Advanced Data Options
  • Redesigned Consume step

You can sign up for the Preview at aka.ms/blog-mb-preview.

Config-based training with generated code-behind files

The Model Builder experience has been revamped! Now when you right-click on your project in Solution Explorer and Add > Machine Learning, the Add New Item Dialog opens, and you can add an ML.NET Model.

New Item Dialog in Visual Studio

After adding your model, the Model Builder UI opens, and a new item (an *.mbconfig file) shows up in the Solution Explorer.

Model Builder UI in Visual Studio

Close up of Solution Explorer in Visual Studio

At any point when using Model Builder, if you close out of the UI, you can double click on the *.mbconfig in Solution Explorer, and it will open the UI again to your last saved state.

After training, two files are generated under the *.mbconfig file:

Solution Explorer expanded with mbconfig in Visual Studio

  • Model.consumption.cs: This file contains the Model Input and Model Output schemas as well as the Predict function generated for consuming the model.
  • Model.training.cs: This file contains the training pipeline (data transforms, algorithm, algorithm hyperparameters) chosen by Model Builder to train the model. You can use this pipeline for re-training your model.
  • Model.zip: This is a serialized zip file which represents your trained ML.NET model.

Previously, these files were added as two new projects (a class library for model consumption code and a console app for the training pipeline). The new experience is similar to adding a new form in a Windows Forms application, where there are code-behind files behind the form and double clicking the form opens the designer.

If you open the *.mbconfig file, you can see that it is simply a JSON file with state information:

  "TrainingConfigurationVersion": 0,
  "TrainingTime": 10,
  "Scenario": {
    "ScenarioType": "Classification"
  "DataSource": {
    "DataSourceType": "TabularFile",
    "FileName": "C:\Desktop\Datasets\yelp_labelled.txt",
    "Delimiter": "t",
    "DecimalMarker": ".",
    "HasHeader": true,
    "ColumnProperties": [
        "ColumnName": "Comment",
        "ColumnPurpose": "Feature",
        "ColumnDataFormat": "String",
        "IsCategorical": false
        "ColumnName": "Sentiment",
        "ColumnPurpose": "Label",
        "ColumnDataFormat": "String",
        "IsCategorical": true
  "Environment": {
    "EnvironmentType": "LocalCPU"
  "Artifact": {
    "Type": "LocalArtifact",
    "MLNetModelPath": "C:\source\repos\ConsoleApp8\ConsoleApp8\MLModel1.zip"
  "RunHistory": {
    "Trials": [
        "TrainerName": "AveragedPerceptronOva",
        "Score": 0.8059,
        "RuntimeInSeconds": 4.4
    "Pipeline": "[{"EstimatorType":"MapValueToKey","Name":null,"Inputs":["Sentiment"],"Outputs":["Sentiment"]},{"EstimatorType":"FeaturizeText","Name":null,"Inputs":["Comment"],"Outputs":["Comment_tf"]},{"EstimatorType":"CopyColumns","Name":null,"Inputs":["Comment_tf"],"Outputs":["Features"]},{"EstimatorType":"NormalizeMinMax","Name":null,"Inputs":["Features"],"Outputs":["Features"]},{"LabelColumnName":"Sentiment","EstimatorType":"AveragedPerceptronOva","Name":null,"Inputs":null,"Outputs":null},{"EstimatorType":"MapKeyToValue","Name":null,"Inputs":["PredictedLabel"],"Outputs":["PredictedLabel"]}]",
    "MetricName": "MicroAccuracy"

This new Model Builder experience brings many benefits. You can:

  • Specify the name of your model and generated code.
  • Have more than one Model Builder-generated model in a solution.
  • Save your state and come back to the last saved state. If you spend an hour training and close out of Model Builder, now you don’t have to start over and can just pick up where you left off.
  • Share the *.mbconfig file and collaborate on the same Model Builder instance via source control.
  • Use the same *.mbconfig file in Model Builder and the ML.NET CLI (coming soon!).

Restructured Advanced Data Options

In the last Model Builder release, we added advanced data options for data loading which gave you more control over column settings and data formatting.

In this release, we added several more options and reorganized the options to make selecting your column settings even easier:

  • Purpose: Choose whether the column is a Feature column, a Label column, or a column to Ignore during training.
  • Data type: Choose whether the data in the column is a String, Single, or Boolean.
  • Categorical: Choose whether the column is categorical or not.

Advanced Data Options in Model Builder

Redesigned Consume Step

We have redesigned the consume step to make a smooth transition from training and evaluating a model to using that model to make predictions in an end-user application.

A code snippet has been provided in the UI which demonstrates how to set up the Model Input as well as how to use the generated Predict function to return the predicted output.

Each Model Input property is filled in with sample data from the first row of your dataset. You can use the copy button in the top right of the box to copy the entire code snippet; then once you paste this code into your end-user application, you can modify the Model Input fields to get real data to feed into your model.

Consume step in Model Builder

Additionally, there is a new Sample project section which generates an application that uses your model and adds the project to your solution. In previous versions of Model Builder, a sample console app was automatically added to your solution; now you can choose whether you want to add a new project to use your model.

Currently, there is only the option to add a console app, but in the future, we plan to add support for Web APIs, Azure Functions, and more.

ML.NET v1.5.5

This release of ML.NET brings numerous bug fixes and enhancements as well as the following new features:

  • New API that accepts double type for the confidence level which helps when you need to have higher precision than an int will allow for. Thank you @esso23 for your contributions!
  • Support for export ValueMapping estimator to ONNX.
  • New API to specify if the output from TensorFlow is batched or not (previously ML.NET always assumed it was a batch amount which caused errors when that wasn’t true).

Check out the release notes for more details.

Virtual ML.NET Community Conference

On May 7th, the 2nd annual Virtual ML.NET Community Conference will kick off with 2 days of sessions on all things ML.NET, and we’re looking for speakers to talk about:

  • MLOps
  • Case studies and real-life use cases
  • Interactive computing with Jupyter
  • ML.NET interop (ONNX)
  • ML.NET and IoT devices
  • ML.NET in F#
  • Big Data and ML.NET
  • A journey from experimentation to production
  • Anything else ML.NET related you can think of!

This is a 100% free event, by the community, for the community.

Published By:

Bri Achtman - Program Manager, .NET

March 15th, 2021

The Blockchain Explained to Web Developers, Part 3: The Truth


After exploring the blockchain theory and using it for real, we now have a better understanding of its strengths and weaknesses. Surprisingly, most of our conclusions are very different from what you will read in the blogosphere. Maybe it’s because we don’t blindly relay the fascination caused by the huge valuations of BitCoin and others. Maybe it’s because the hard truth about the blockchain is that it’s not ready yet. Read on to understand our take on the blockchain, based on strong evidence.

The Technology Is Not Mature Enough

As explained in detail in the previous post in this series, developing Decentralized Apps over a blockchain is a pain. The development community is small, available code snippets don’t work, public tutorials are outdated, the libraries are crippled with bugs, developer tooling is lacking, bugs are silent, etc.

It’s not that the Ethereum developers and community are bad ; they’re amazing, and they’re pouring a lot of time and expertise into their tools. But building a blockchain framework is a huge amount of work, and they’re only halfway through. Ethereum hasn’t reached the point of usability yet. I’m confident that this will change in the future, but I don’t know if it’s a matter of months or years.

Tip: We haven’t developed DApps for Bitcoin, but I’ve heard it’s worse. Instead of using a JavaScript-like language (Solidity) for smart contracts, you must use an assembly-like language, which isn’t even Turing-complete. Yikes.

The consequence is that developers don’t want to work on blockchain projects - they find it very frustrating. If you force them to work with a technology they hate, they will leave. Since it’s extremely hard to find skilled developers these days, you should think twice before taking a chance on the blockchain.

The second consequence is that it’s impossible to estimate the time it will take to build a project on the blockchain. If you can’t estimate your costs, good luck building a Business Model on the blockchain.

Smart Contracts Can’t Call APIs

In our blockchain experimentation, everything a bit “smart” in the contract had to be moved to a plain old web service running outside of the blockchain, in a trusted environment. For instance, a smart contract can’t figure out if the person asking for an ad placement is the author of a pull request, because a smart contract can’t call the GitHub API. As a consequence, our smart contract keeps only a very minimal amount of logic, becoming, in fact, a dumb contract. It’s not because we wanted to, it’s because we couldn’t do otherwise.

By design, a blockchain is deterministic. That means that if you take the entire history of blocks, and replay it locally, you should end up with the same state as every other node. This forbids the call to external APIs, where responses may change over time, or according to who calls them.

Blockchains are walled gardens. You can execute a contract from the outside world, but a contract itself can’t require data from a source outside of the blockchain. If a smart contract needs external data, someone must push the data to the blockchain first. There is an effort to ease this process through a concept called Oracles. But Oracles need a reputation system and governance. So much for fully-automated contracts and disintermediation.

In the real world, very few applications work in isolation. All the applications we’ve built for the past 3 years relied on external APIs - for identity management, payment, live data source, image processing, storage, etc. The limited capabilities of smart contracts make them useless in real world situations.

You Need A PhD to Understand It

If you read through the first blog post of this series, you probably think that you have a good basic understanding of the blockchain. Now, go and read this article. I’m an average engineer with only 20 years of experience in Web Development, and I couldn’t understand anything after the Jurassic Park reference. Terms like “two-way-pegged blockchains”, “pre-determined Host Oracle Contract”, and sentences like “The M-S result, combined with our inability to feed (non-BB) a revelation mechanism, means that Oracles are out” make me fell like a first grader.

The blockchain concept is complex. Existing implementations rely on rare design patterns, that you don’t learn in college. The blockchain vocabulary is kabbalistic.

Developing decentralized apps on top of blockchains requires understanding too many complicated concepts to fit in an average developer’s brain. My opinion is that there are not enough highly skilled programmers to support the revolution promised by the blockchain. And there will never be, as long as it’s so hard to understand.

As a consequence, most Decentralized apps are very buggy. A recent article stated that smart contracts contain 1 bug every 10 lines of code, making Ethereum “candy for hackers”. It wouldn’t be such a big deal if fixing bugs was easy. Unfortunately, as we explained in the previous post, you can’t update a smart contract. You have to create a new contract, transfer all the data and pointers from the old contract to the new one, and wait for the blockchain to propagate the change. The old buggy contracts and transactions remain in the blockchain forever.

Developer Power

The blockchain authors suggest using “code as law”. This also means “bugs as law”, as every software contains bugs. These bugs can be used by smart developers (criminals, the NSA, etc.) to avoid playing by the rules. Bugs are very common, even in popular open-source projects. Bitcoin, for instance, suffered several critical bugs leading to “cybertheft”. So leaving the keys to developers also means giving extraordinary power to the mean developers.

I don’t want to go all FUD (Fear, Uncertainty and Doubt) on you, but the possible scenarios of a society governed by machines don’t all finish with a happy ending in my mind.

When machines control the world

And even if we don’t consider mean developers, giving the power to good developer is dangerous, too. The problem is that developers are irresponsible (no harm intended - I’m a developer myself). It’s not that they’re childish, it’s that nobody ever taught them to write the law.

Also, developers are not elected by the people. If you don’t agree with the direction that Bitcoin takes (favoring speculation rather than practical applications) too bad for you - there is nothing you can do to change that. This is currently happening: the Bitcoin network currently suffers a severe crisis, because of the disagreements between a few core developers.

The decisions of half a dozen developers may cause the collapse of a billion dollar market capitalization. But nobody will hold them accountable in case of failure.

Waste of Resources

A blockchain is not cost-effective at all. In fact, it’s a huge waste of resources.

Take data replication for instance. The blockchain replicates all transactions across all nodes. Engineers have long invented replication strategies with better space efficiency. Compare the Blockchain with RAID6 disk clustering for instance:

In a Blockchain network, 10 nodes of 1GB each allow for a total replicated data volume of 1GB. You can loose up to 9 nodes in the network, and yet be able to recover the entire data.

In a RAID6 pool, 10 hards disks of 1GB each allow for a total replicated data volume of 8GB. You can loose up to 2 HDD in the pool, and yet be able to recover the entire data.

Mining nodes require very expensive hardware, with high end GPU cards and a huge amount of memory.

And it’s not just about buying expensive hardware. 99.99% of the computing is just wasted. All miners compete to mine a block by running expensive Math challenges. In Bitcoin, only one node every 10 minutes wins, and is actually useful to the chain by creating a block. The computation done by all the other nodes is thrown away.

The Ethereum blockchain is trying to fix that: they plan to switch from a proof-of-work consensus algorithm to a proof-of-stake, which is much less resource intensive. But proof-of-stake also has drawbacks, such as giving more power to people or companies owning high amounts of cryptocurrency. Besides, it’s far from ready yet (expect it in at least a year from now).

When machines control the world

This waste of storage, CPU and memory translates into a huge waste of energy. According to a bitcoin mining-farm operator, energy consumption totaled 240kWh per bitcoin in 2014 (the equivalent of 16 gallons of gasoline). Mining farms are a distributed engine turning electricity into heat. A blockchain is, in short, an expensive radiator. Energy efficiency is a big deal in a globally warming planet.

Very Expensive

Who pays for all the wasted energy? The companies that publish and use smart contracts. Yes, that’s you, if you intend to run a business on the blockchain. When you pay for a transaction on the blockchain, you also pay 99.99% of the network running at full speed for nothing. That makes blockchain transactions expensive.

A million dollars in bank notes

An average BitCoin transaction requires a fee of BTC 0.0002 ($0.11). This price is rising. It’s not really cheaper than a bank transaction fee (unless you consider a transfer across two countries with different currencies, of course).

For ZeroDollarHomepage, executing a 10-lines script on Ethereum method costs about one cent ($0.01). That’s insanely expensive. Amazon Lambda, for instance, costs $0.0000002 per request (after the first million requests each month).

It’s normal to pay for hosting costs when you use a platform, but the Blockchain costs are orders of magnitude higher than the most expensive PaaS.

Volatility and Speculation

You could say that the blockchain cost isn’t such a big deal, as long as people are willing to use the network and pay for transactions. It’s a question of supply and demand, and the demand for blockchain and cryptocurrencies is currently high enough to make it profitable. But this high demand leads to speculation, and therefore the price of computing and storage in a blockchain (any blockchain) is highly volatile.


Some analyst compare Bitcoin to a Ponzi Scheme, and predict that the market value will collapse once general interest disappears.

If we build a business based on the Ethereum’s blockchain, most of our expenses will be in Ether. If we don’t mine it ourselves, we’ll have to pay for that Ether in real money. But since the USD value of Ether may vary tenfold within a year, our business can move from profitable to worthless in the same timeframe. And we can’t do anything against it. On the other hand, if we mine ourselves, what is currently affordable (running a small server to cover expenses in Ether) might become very expensive once very large mining farms move from Bitcoin to Ethereum.

The high volatility of cryptocurrencies forbids any long-term profitable business built on the blockchain - except speculation.

Slow As Hell

Compared to many other innovations based on computers and networks, the blockchain is very slow. Experts say that you should wait 6 blocks to make sure that a transaction is legit. This means more than 1 minute in Ethereum, or more than 1 hour in Bitcoin.

In a traditional ad server, scheduling an ad takes about 100ms. If you’ve used our ZeroDollarHomepage Ad Server, you probably had a very different experience: Scheduling an ad takes about a minute. The network transport and replication accounts for a small share of that duration ; most of the time is spent waiting for the network to mine the transaction, and add a few more blocks after that. But all in all, the Ethereum blockchain is several orders of magnitude slower than traditional computing.


For end users, every second counts. The Web Performance Optimization trend focuses on improving revenue by earning one or two seconds in download time. Betting on a technology that requires a transaction to be acknowledged by the entire world isn’t the best way to make business.

Free Market and Anarchy

One of the promises of the blockchain is to liberate markets that still require an intermediary. No more lawyers, bankers, or bookmakers. A great opportunity for new businesses?

Except these intermediaries currently report criminal activities to the authorities (governments and law enforcement agencies). If you remove the intermediaries, you also remove the police, and you let criminals proliferate. The first bitcoin application at scale was called The Silk Road. It was an online marketplace for everything illegal: drugs, weapons, child pornography, etc. Not to mention the ability to use bitcoins for tax evasion.

Even the proponents of free market economy recognize that a certain level of regulation is necessary to avoid total chaos. Running a business in a land full of criminals with no police isn’t profitable - unless you’re a criminal, too. For instance, the Mt. Gox Bankrupcy in 2014 cost about $450 million to BitCoin users.

Just like it took a long time for governments to control the Internet (which was, and remains, a haven for criminals), it will take a long time for our lawmakers to control the anarchy unleashed by blockchains. The blockchain may carry the promise of a better future in the long term, but for the near future, you’d better be armed.

Do You Really Need A Blockchain?

A large share of the hype around the blockchain comes from people who don’t really understand its shortcomings. They would probably use another solution is they were better informed. Here are a few bad reasons why you should probably not choose the blockchain technology.

You can use a private blockchain Nearly 80% of the blockchain projects I hear about, especially in finance, are based on private blockchains. This completely defeats the main purpose, which is to get an agreement between non-trusted parties. If a project needs runs on a private blockchain, then only trusted parties can join it, and you don’t have a trust problem. In a trusted network, there are many, many other tools to share a ledger of facts - all much better optimized than the blockchain (for instance: a web service).

It offers a way to reach distributed consensus It does, but only if this consensus can be written as code. For instance, a company working with music rights distribution recently contacted us to build an international platform for artist retribution on the blockchain. Except that when two countries disagree on how to pay right holders, they both have valid contracts. Only a court can decide which contract wins. No smart contract can replace that. You must have clear governance rules that already work before trying to automate them in a blockchain.

It’s secure Asymmetric cryptography is one of the blockchain’s strengths. However, the blockchain technology, just like any other, is safe only until someone finds a vulnerability. It has already happened in the past. The computer science behind the blockchain is so complex that very few developers can contribute or review the code. Consider smart contracts and blockchains as relatively less secure than, say, TSL on the web (through HTTPS). Of, and even if the software works perfectly, it doesn’t prevent fraud. Remember the double spend problem from our first post? It turns out people regularly try that in blockchains (see the latest 200 double spends in the Bitcoin blockchain)

It’s transparent Granted, all transactions are public, and expose location and IP address. But no personal information ever transits - only anonymous hashes. Even the creator of Bitcoin is a mystery. So blockchain transparency doesn’t prevent crime or fraud. Also, transparency is usually an inconvenient for businesses. Are you willing to bet your business on a technology that lets everyone track all your transactions, and exposes your code to hackers?

Data is replicated and safe Sure, but with the least cost effective replication strategy. Amazon S3 replicates every bit of data at least 3 times with 100% uptime, for a fraction of the price. And if you actually need full transaction history, use an event store.

It connects anonymous peers But if it’s only for a shared storage (i.e. if you don’t need fact ordering), then regular peer-to-peer network protocols like BitTorrent are enough.

It’s hip I can’t argue with that: yelling the word “blockchain” out loud is currently a great way to grab an innovation budget. However, many of the shining products that pretend to run on the blockchain are merely powerpoint presentations. Besides, you’ll get better results with many other technologies. Not to mention that the word blockchain also evokes money laundering, tax fraud, and pornography.

If you want to build your business on the blockchain, be certain that you need it, and that it will be really useful for your use case.


Blockchains are a very smart idea, with huge possible implications. But are the current implementations ready to power the disruptive applications of the next decade?

On the technical side, some elementary features are simply not feasible. Blockchains are not efficient enough, not enough developer-friendly, and they give too much power to a small league of extraordinary developers without enough political and economical background.

On the business side, the blockchain is moving too fast, it’s expensive, and often overkill. Costs may vary tenfold for no reason. Building a business on such an unstable platform is incredibly risky.

My take is that we have to wait. The blockchain isn’t ready yet. It needs more maturity, another killer app than a speculation engine, a larger developer community, more ecological and economical responsibility. How long will it take? Maybe a year or two? Nobody can tell that.

To be honest, this conclusion surprised me. Most of the publications about the blockchain suggest the opposite. They say “it’s time”, “don’t miss the train”, or “the giant businesses of the next decade are being built on the blockchain right now”. Maybe they are wrong, or maybe we are wrong. We’ve tried to argument this analysis with strong evidence. If you have a different opinion, please voice your comment below.

We’ll be following the developments in the different blockchain projects closely. Make sure you follow this blog for related news!

The Blockchain Explained to Web Developers, Part 2: In Practice

Published on 20 May 2016 by Gildas Garcia and Kevin Maschtaler

Edmonton's first digital billboard?

Is the blockchain a revolution? The technology that powers Bitcoin sure has the potential to disrupt the entire Internet, as we explained in a previous post. But how can you, a developer, use the blockchain to build applications? Are the tools easy to use, despite the complexity of the underlying concepts? How good is the developer experience?

We wanted to find out, and there is no better tutorial than developing an app from scratch. So we’ve made a simple decentralized ad server called Zero Dollar Homepage, powered by blockchain. This is the story of our experience. Read on to learn how hard the blockchain is for developers today.

Application Concept

The blockchain shines when it replaces intermediaries. We chose to focus on Ad Platforms, which are intermediaries between announcers (who buy visibility) and content providers (who sell screen real estate). Our project was to build a decentralized ad platform running on the blockchain.

Since the famous Million Dollar Homepage experiment, innovating in the field of paid ads can’t make you rich anymore.

Instead, we chose to build a tool that allows to display ads for free - a Zero Dollar Homepage. For free, but not for nothing: advertisers exchange ad visibility for open-source contributions. So we’ve built a decentralized app to manage how ads display on a particular page. Advertisers need to take up a coding challenge to be able to put their ads on this page.

User Workflow

In concrete terms, whenever we merge a Pull Request (PR) on one of marmelab’s open-source repositories, a GitHub bot comments on the PR, and invites the PR author to publish their ad on the ad platform admin panel.

Following the link contained in the comment, the PR author is asked to sign in with their GitHub credentials. Then, they can upload an ad - in fact, a simple image. This image is added to the list of images uploaded by other PR authors, in chronological order.

Each day at midnight, an automated script takes the next image in line (using a FIFO ordering), and displays it on http://marmelab.com/ZeroDollarHomepage/ for the next 24 hours.

Note: The entire process requires no intermediary, but in order to avoid the display of adult imagery on our site, we validate the uploaded images through the Google Vision API before putting them online.


Here is how we separated responsibilities in each of the 4 use cases of an ad platform:

  1. Open-source contributor notification Whenever an open-source PR gets merged on one of our repositories, GitHub notifies the admin app with the PR details. The app publishes a comment on the PR to notify the contributor. The comment contains a link back to the admin app, with the PR details.
  2. Claim and image upload Following the comment link, the contributor goes to the admin app. He must sign in with his GitHub credentials to be authenticated. The admin app then calls GitHub to grab the PR details, and to check that the contributor is actually the PR author. If it’s OK, the admin app displays an image upload form. When the contributor uploads an image, the admin app pushes the PR id to the blockchain, and uploads the image to a CDN (named after the PR id). The admin app displays the approximate date of publication of the image based on the number of valid PRs with an image still waiting in the blockchain.
  3. Ad placement Every 24 hours, a cron asks the blockchain for the next PR not yet displayed. The blockchain marks this PR as displayed and sends the ID. The cron renames the image named after the pr ID to “current image”.
  4. Ad display Each time a visitor wants to display the ad in ZeroDollarHomepage, it asks the CDN for the current image. It happens to be the latest published ad from the blockchain, which remains displayed at least 1 day (and until another contributor claims a PR).

This might seem surprising, as the blockchain plays a very small part in the process. But we quickly realized that the entire code of the ad platform couldn’t live in the blockchain. In fact, blockchains have very limited capabilities in terms of connectivity to the Internet, and processing power. So we delegated only the crucial ad placement tasks to the blockchain:

  • Register a pull request by an authenticated contributor
  • Get the last non displayed pull request, and mark it as displayed

Other tasks ended up in the admin app, outside of the blockchain, for various reasons:

  • Register a pull request from a webhook Registering a pull request before it’s been claimed is useless, since the contributor may never claim it. Besides, storing data in the blockchain isn’t free, so we store only what we have to store. The downside is that any PR on our public repositories, including those created before this experiment, are eligible for the next step.
  • Notify the user by posting a comment to GitHub A smart contract can’t call an external API, so it’s just not possible. Instead, we delegated this task to the admin app.
  • Verify a claimed PR’s author Again, a smart contract can’t call the GitHub APIs. Instead, we moved this logic to the admin app, and made it a prerequisite before calling the blockchain.
  • Store the Image In theory, you can store pretty much anything in the blockchain, including images. In practice, images cost a lot to store, and we didn’t manage to store more than one “table” (array of data) in our smart contract.
  • Update the displayed ad to the next in line A blockchain has no equivalent of the setTimeout function, or cron jobs. You might however execute some code every x blocks but it’s not related to time. Instead, we used a cron-like library on our API.

Research, documentation and first attempts

As we explained in a previous post, they aren’t many good choices when choosing a blockchain network. So we chose Ethereum.

We quickly hit our first wall. Until a few weeks ago, you couldn’t play with the Ethereum blockchain without buying Ether first, even for simple tests. Also, Ethereum didn’t really allow private blockchains in its former version (named Frontier), which made development very complicated. Anyone accessing the Ethereum network might call your test contracts. More importantly, the documentation is a volunteer initiative, and was not in sync with the development state.

Note: Ethereum bumped their version since we developed the application, switching fromFrontier to Homestead. The Ethereum community improved the documentation quality for Homestead.

Despite these shortcomings, we managed to register three nodes on the Ethereum network across Nancy, Paris and Dijon, and to share a ping between those nodes.

In the course of our documentation search, we eventually found the Eris documentation. Eris did an excellent job at explaining Blockchains and contracts. Moreover, they especially built a layer on top of Ethereum, and open-sourced a bunch of tools to ease the process of developing smart contracts.

eris is command line tool you can use to initialize any number of local blockchains you need.

Smart Contract Implementation

A smart contract is very similar to an API. It has a few public functions which might be called by anyone registered on the blockchain network. Unlike an API, a smart contract cannot call external web APIs (a blockchain is a closed ecosystem). A smart contract may however call other smart contracts, provided it knows their address.

As with an API, the public functions are only the tip of the iceberg. A contract might be in fact composed of many private functions, variables, etc.

Smart contracts are hosted in the blockchain in an Ethereum-specific binary format, executable by the Ethereum Virtual Machine. Several languages and compilers are available to write contracts:

At marmelab, we code a lot in Javascript, so we chose to use Solidity. Solidity contracts are stored in .sol files.

The Zero Dollar Homepage Contract

The Zero Dollar Homepage contract stores the claimed pull-requests, and a queue of requests to display. The first version of the Solidity contract looked like this:

// in src/ethereum/ZeroDollarHomePage.sol
contract ZeroDollarHomePage {
    uint constant ShaLength = 40;

    enum ResponseCodes {

    struct Request {
        uint id;
        string authorName;
        string imageUrl;
        uint createdAt;
        uint displayedAt;

    // what the contract stores
    mapping (uint => Request) _requests; // key is the pull request id
    uint public numberOfRequests;
    uint[] _queue;
    uint public queueLength;
    uint _current;
    address owner;

    // constructor
    function ZeroDollarHomePage() {
        owner = msg.sender;
        numberOfRequests = 0;
        queueLength = 0;
        _current = 0;

    // a contract must give a way to destroy itself once uploaded to the blockchain
    function remove() {
        if (msg.sender == owner){

    // the following three methods are public contracts entry points

    function newRequest(uint pullRequestId, string authorName, string imageUrl) returns (uint8 code, uint displayDate) {
        if (pullRequestId <= 0) {
            // Solidity is a strong typed language. You get compilation errors when types mismatch
            code = uint8(ResponseCodes.InvalidPullRequestId);

        if (_requests[pullRequestId].id == pullRequestId) {
            code = uint8(ResponseCodes.PullRequestAlreadyClaimed);

        if (bytes(authorName).length <= 0) {
            code = uint8(ResponseCodes.InvalidAuthorName);

        if (bytes(imageUrl).length <= 0) {
            code = uint8(ResponseCodes.InvalidImageUrl);

        // store new pull request details
        numberOfRequests += 1;
        _requests[pullRequestId].id = pullRequestId;
        _requests[pullRequestId].authorName = authorName;
        _requests[pullRequestId].imageUrl = imageUrl;
        _requests[pullRequestId].createdAt = now;

        queueLength += 1;

        code = uint8(ResponseCodes.Ok);
        displayDate = now + (queueLength * 1 days);
        // no need to explicitly return code and displayDate as they are in the method signature

    function closeRequest() returns (uint8) {
        if (queueLength == 0) {
            return uint8(ResponseCodes.EmptyQueue);

        _requests[_queue[_current]].displayedAt = now;
        delete _queue[0];
        queueLength -= 1;
        _current = _current + 1;
        return uint8(ResponseCodes.Ok);

    function getLastNonPublished() returns (uint8 code, uint id, string authorName, string imageUrl, uint createdAt) {
        if (queueLength == 0) {
            code = uint8(ResponseCodes.EmptyQueue);

        var request = _requests[_queue[_current]];
        id = request.id;
        authorName = request.authorName;
        imageUrl = request.imageUrl;
        createdAt = request.createdAt;
        code = uint8(ResponseCodes.Ok);

For this first attempt, we used the Eris JS libraries to communicate with our blockchain. Instanciating a contract from a Node.js file turned up to be as simple as:

import eris from 'eris';

function getContract(url, account) {
    const address = // Read address file stored on disk by the eris CLI;
    const abi = // Read abi file stored on disk by the eris CLI;
    const manager = eris.newContractManagerDev(url, account);
    return manager.newContractFactory(abi).at(address);

And calling it wasn’t difficult either:

function* newRequest(pullrequestId, authorName, imageUrl) {
    const contract = getContract(url, account);
    // First gotcha, when a function returns several named variables, they are returned as an Arrays
    // Second gotcha, numbers are returned as instances of BigNumber, do not forget to convert when standard numbers are expected
    const [codeAsBigNumber, displayDateAsBigNumber] = yield contract.newRequest(pullrequestId, authorName, imageUrl);
    const code = codeAsBigNumber.toNumber();

    if (code !== 0) {
        throw new Error(getErrorMessageFromCode(code));

    // Return the displayDate for UI confirmation screen
    return displayDate.toNumber();

For more information about the Eris JS binding libraries, please refer to Eris documentation.

Unit Testing Contracts

We love Test Driven Development, and one of the first question we had was: how can we test a Solidity smart contract?

The Eris guys made a tool for that, too: sol-unit. It runs a new local blockchain network for each test, in a docker container (which ensures each test run in a clean environment), and executes the test. Tests are written as a contract, too. Neat!

Well, not so fast. sol-unit is an npm package, and to use the testing functions (assertions, etc.), we had to import the contract supplied by this package in our testing contracts. For that, there is a simple Solidity syntax:

import "../node_modules/sol-unit/.../Asserter.sol";

So far so good… or not. We hit a strange case when compiling our contracts. Apparently, you can’t import contracts with such a path. We ended up adding a command in our testmakefile target to copy those sol-unit contracts in the same folder as ours. After that, running sol-unit was simple and we started coding.

	@cp -f ./node_modules/sol-unit/contracts/src/* ./src/ethereum/

	solc --bin --abi -o ./src/ethereum ./src/ethereum/ZeroDollarHomePage.sol ./src/ethereum/ZeroDollarHomePageTest.sol

test-ethereum: copy-sol-unit compile-contract
	./node_modules/.bin/solunit --dir ./src/ethereum

Running a Test Blockchain

Running a blockchain and deploying our contract to it was as simple as following the Eris documentation. We managed to resolve the few troubles we met using a bunch of commands that we integrated in our makefile. The whole process of running a new blockchain with our contract looks like this:

  • Reset any running eris docker containers, and remove some temporary files
  • Start the eris key service
  • Generate our account key, and store its address in a convenient file to be loaded later by the JS API
  • Generate the genesis.json, which is the “block 0” of the blockchain
  • Create and start the new blockchain
  • Upload the contract to the blockchain and save its address in order to call it when we need it

After a few days of work, we were able to run the contracts on a local Eris blockchain.

From Eris to Ethereum

At this point, we wanted to try out our contracts on a local Ethereum blockchain.

To communicate with contracts inside the Ethereum blockchain, we had to use the Web3 libraries. We learned a lot while trying to use them. We realized that eris was hiding a lot of the underlying complexity.

First, our initial assumption that a contract is similar to an API was not correct. We had to distinguish functions that were only reading data from the blockchain, and functions that were writing data to it.

The first kind (read-only functions) would return the resulting data asynchronously, just like an API would do. The second kind (write functions) would only return a transaction hash. The expected side effects of a write function (changes inside the blockchain) wouldn’t be effective until the corresponding blocks would be mined, which could take some time (from ten seconds to one minute in the worst case). Moreover, we haven’t been able to make those writing functions return values, so we had to change our solidity code to call a write function first, then call a read function to get the results.

We also discovered events, which can be used to be notified when something happens in a smart contract. The smart contract is responsible for triggering the events. They look like this with solidity:

event PullRequestClaimed(unit pullRequestId, uint estimatedDisplayDate);

And they can be triggered from any of the smart contract functions, like this:

PullRequestClaimed(pullRequestId, estimatedDisplayDate);

Those events are stored permanently in the blockchain. That means we could use the blockchain as an event store. It might be the easiest way to determine if a call to a function has been successfully executed: the smart contract may trigger an event at the end of its process with failure reasons, results of computation, etc… It’s worth noting that some integration packages for Meteor are already available.

Eventually, we refactored our smart contracts to be a lot simpler in order to get almost the same features. We had to get rid of the mappings (which we haven’t been able to use - our transactions weren’t mined by the Ethereum network for some reason).

The solidity language may be close to JavaScript, it is still very young and incomplete. Arrays don’t have the functions we’re used to work with in JavaScript (not even indexOf), and strings don’t have any functions. This might be addressed by some community efforts in the near future.

The Ethereum implementation looks like this:

// in src/ethereum/ZeroDollarHomePage.sol
contract ZeroDollarHomePage {
    event InvalidPullRequest(uint indexed pullRequestId);
    event PullRequestAlreadyClaimed(uint indexed pullRequestId, uint timeBeforeDisplay, bool past);
    event PullRequestClaimed(uint indexed pullRequestId, uint timeBeforeDisplay);
    event QueueIsEmpty();

    bool _handledFirst;
    uint[] _queue;
    uint _current;
    address owner;

    function ZeroDollarHomePage() {
        owner = msg.sender;
        _handledFirst = false;
        _current = 0;

    function remove() {
        if (msg.sender == owner){

    function newRequest(uint pullRequestId) {
        if (pullRequestId <= 0) {

        // Check that the pr hasn't already been claimed
        bool found = false;
        uint index = 0;

        while (!found && index < _queue.length) {
            if (_queue[index] == pullRequestId) {
                found = true;
            } else {

        if (found) {
            PullRequestAlreadyClaimed(pullRequestId, (index - _current) * 1 days, _current > index);

        PullRequestClaimed(pullRequestId, (_queue.length - _current) * 1 days);

    function closeRequest() {
        if (_handledFirst && _current < _queue.length - 1) {
            _current += 1;

        _handledFirst = true;

    function getLastNonPublished() constant returns (uint pullRequestId) {
        if (_current >= _queue.length) {
            return 0;

        return _queue[_current];

The process for claiming a pull request and returning the estimated display date evolved to become:

// make a [transaction](https://github.com/ethereum/wiki/wiki/JavaScript-API#web3ethsendtransaction) call to our smart-contract write function
contract.newRequest.sendTransaction(pullrequestId, {
    to: client.eth.coinbase,
}, (err, tx) => {
    if (err) {
        throw error;

    // wait for it to be mined using [code](https://github.com/ethereum/web3.js/issues/393) from [@croqaz](https://github.com/croqaz)
    return waitForTransationToBeMined(client, tx)
        .then(txHash => {
            if (!txHash) throw new Error('Transaction failed (no transaction hash)');

            // get its receipt which might contains informations about event triggered by the contract's code
            // this function might also check wether the transaction was successful by analyzing the receipt for ethereum specific error cases (insufficient funds, etc.)
            return getReceipt(client, txHash);
        .then(receipt => {
            // parse those logs to extract only event data
            return parseReceiptLogs(receipt.logs, contractAbi));
        .then(logs => {
            if (logs.length === 0) {
                throw new Error('Transaction failed (Invalid logs)');

            const log = logs[0];

            if (log.event === 'PullRequestClaimed') {
                // timeBeforeDisplay is a BigNumber instance
                return log.args.timeBeforeDisplay.toNumber();

            if (log.event === 'PullRequestAlreadyClaimed') {
                const number = log.args.timeBeforeDisplay;

                if (log.args.past) {
                    // timeBeforeDisplay is a BigNumber instance
                    return number.negated().toNumber();

                // timeBeforeDisplay is a BigNumber instance
                return number.toNumber();

            if (log.event === 'InvalidPullRequest') {
                throw new Error('Invalid pull request id');

And with this code, our decentralized app worked in a local Ethereum network.

Deployment to Production

Running our application in a local environment was a challenge, but deploying it to production, in the real Ethereum network, was a battle.

There are a few gotchas to be aware of. The most important one is that contracts are immutable in code. This means that:

  • A contract that you deploy to the blockchain stays there forever. If you find a bug you your contract, you can’t fix it - you have to deploy a new contract.
  • When you deploy a new version of an existing contract, and any data stored in the previous contract isn’t automatically transferred - unless you voluntarily initialize the new contract with the past data. In our case, fixing a bug in the contract actually wipes away recorded PRs (whether already advertised, or waiting for ad display).
  • Every contract version has an id (for instance, the current ZeroDollarHomepage contract is 0xd18e21bb13d154a16793c6f89186a034a8116b74). Since past versions may contain data, keep track of past contract ids if you don’t want to lose the data (this happened to us, too).
  • As you can’t update a contract, you can’t rollback an update either. Make really sure that your contract works before redeploying it.
  • When you deploy a new version of an existing contract, the old (buggy) contract can still be called. Any system outside of the blockchain referencing the contract (like our Node admin app in Zero Dollar Homepage) must be updated to point to the new contract. We forgot to do it a few times, and scratched our head desperately to understand why our new code didn’t run.
  • Contracts authors can kill their contract if they include a suicide call in the code. But all the existing transactions of the contract remain in the blockchain - forever. Also, make sure that the kill switch deals with the remaining ether in the contract if you don’t want it to disappear.

Another gotcha is that every contract deployment and write operation in the blockchain costs a variable amount of ether. We managed to get 5 ETH (more about getting ether below), but we had no idea how much we would need to deploy our contract, or calling a transaction. It’s harder to test when each failed test costs money.

For the Node.js part, we decided to run it on an AWS EC2 instance, like most of our projects. To do so, we had to:

  • Run an Ethereum node on the server
  • Download the entire blockchain to this server
  • Unlock an account with some Ether on the node
  • Deploy our application and link it to the node
  • Register our smart contract into the blockchain through the node

Make sure your blockchain node server has plenty of storage. The current size of the blockchain is about 15GB. The default volume size on an EC2 instance is 8GB (sigh). We had many troubles because we hadn’t downloaded the entire chain (but we didn’t realize it immediately). For instance, we had an account with 5 ETH, but for a long time the system responded as if we hadn’t unlocked our account, or as if we had no ether. Downloading the rest of the chain fixed this issue.

Likewise, unlocking our precious account containing 5 ETH was not an easy task. We did not want to hardcode our passphrase in the application, and we wanted to run the node with supervisord to ease the deployment. We finally found a way that allowed us to change the configuration without exposing our passphrase with the following supervisordconfiguration:

command=geth --ipcdisable --rpc --fast --unlock 0 --password /path/to/our/password/in/a/file

One final security note: The Remote Procedure Call (RPC) port of the blockchain is 8545. Do not open this port on your EC2 instance! Anyone knowing the instance IP could control your Ethereum node, and steal your ether.

Ether and Gas

As explained in our first post on the blockchain, deploying and calling a contract in the Ethereum blockchain isn’t free. Since a blockchain is expensive to run, any write operation must be paid for. In Ethereum, the price of calling a write contract method depends on the complexity of the method. Ethereum comes with a list of Gas Fees, which tells you how much Ether you should add to a contract call to have it executed.

In practice, that’s a very low amount of Ether, a fraction of a single Ether. The Ethereum blockchain introduced another currency for running contracts: Gas.

1 Gas = 0.00001 ETH 1 ETH = 100,000 Gas

The Gas to Ether conversion rate will vary in the future according to the supply of computing power, and the computation demand.

Charging a fee to process a transaction isn’t compulsory, but recommended. The Ethereum documentation says: “Miners are free to ignore transactions whose gas price is too low”. However, a mined block always give 5 ETH to the successful miner.

To call our own contracts, the Ethereum blockchain requires between 0.00045 and 0.00098 Ether (the price depends on the gas price and the gas used by the transaction).

How do you get Ether and Gas? You can buy Ether (mostly by exchanging Bitcoins), or you can mine it. In France, where we live, buying Bitcoins or Ether requires almost the same procedure as opening a bank account. It’s slow (a few days), painful, and depends on exchange rates fixed by offer and demand.

Mining Ether

So we decided to mine our Ether. That’s a good way to see if mining is profitable on Ethereum or not. We spawned a heavy Amazon EC2 instance, with strong GPU computing power (a g2.2xlarge instance). The price of this instance is 17$ per day. We installed Ethminer, and started our node. We quickly had to beef up the instance even more, because of high memory and storage requirements. The first thing a node does when it joins a blockchain is to download the entire history of past transactions. That takes a huge amount of storage: over 14GB for the blockchain’s history, and about 3GB for the Ethash Proof of Work.

Once our Ethereum node started, we had to mine for 3 days to create a valid block:

As a reminder, the Ethereum blockchain mines one block every 10 seconds. Mining a block brings up 5 Ether, which sell for roughly $55. The running cost for our beefy EC2 instance for these 3 days was about $51. All in all, it was cheaper to mine Ether on AWS than to buy it. But we were very lucky: since we mined our block, the network’s difficulty was multiplied by three.

How long can we run the ZeroDollarHomePage with 5 Ether? Let’s make the computation.

The Zero Dollar Homepage workflow implies one transaction per day, plus one transaction per claimed PR. Supposing contributors claim one PR per day, the yearly price in ether for running the platform would be at most 365 * 2 * 0,00098 = 0.72 ETH. With 5 ETH, we should normally be able to run the platform for almost seven years.

As you see, running a contract in Ethereum isn’t free, but at the current price of Ether, it’s still cheap. However, the Ether value varies a great deal. Since mining Bitcoin is becoming less and less profitable, some large Bitcoin mining farms switch to Ethereum. This makes mining harder, and makes Ether more expensive every day.

Final Surprise

Finally, our smart contract ended up working fine in our real world Ethereum node hosted on EC2.

But by the time we got there, Ethereum released their Homestead version, which brought a lot of new things and broke our code entirely. It took us about a week to understand why, through trial and error, and fix the code that wasn’t compatible anymore for obscure reasons.

Tip: The Homestead release documents a hidden Ethereum feature, private networks, to ease development. The lack of private networks was one of our reasons to use Eris in the first place.

The ZeroDollarHomePage platform is now up and running again. You can use it by opening a pull request on one of marmelab’s open-source repositories on GitHub, see the ads currently displayed on http://marmelab.com/ZeroDollarHomepage/, or browse the code of the application on marmelab/ZeroDollarHomePage. Yes, we’re open-sourcing the entire ad platform, so you can see in detail how it works, and reproduce it locally.


The Ethereum developer experience is very bad. Imagine that you have no logs and no debug tools. Imagine that the only way to discover why a program fails is to echo “I’m here” strings every line to locate the problem. Imagine that sometimes (e.g. in Solidity contracts), you can’t even do that. Imagine that a program that works perfectly in the development environment (where you can add debug statements) fails silently in the production environment (where you can’t). That’s the developer experience in Ethereum.

If you store data in your smart contract, there is no built-in way to visualize the current state of this data after a transaction. That means you need to build your own visualisation tool to be able to troubleshoot errors.

The tools available to track Ethereum contracts and transactions are:

For instance, here is how our contract looks in etherscan:

Each transaction (call to a contract method) is logged there, together with a trace of the contract execution… in machine language. Apart from making sure your call actually gets to the contract, you can’t use it for debugging.

Also, these tools can only monitor the public Ethereum network. Unfortunately, you can’t use them to debug a local blockchain.

If you have ever seen Bitcoin transaction auditing sites, don’t expect the same level of sophistication for Ethereum. Besides, the bitcoin network only has one kind of transaction, so it’s easier to monitor than a network designed to run smart contracts.


And that’s not all: the Ethereum documentation is not in sync with the code (at least in the Frontier version), so most of the time we had to look at the libraries to try to understand how we’re expected to code. Since the libraries in question use a language that no one uses (Solidity), good luck figuring out how they work. Oh, and don’t expect help from Stack Overflow, either. There are too few people like us who dared to implement something serious to have a good community support.

Let’s be clear: we are not criticizing the Ethereum community for their lack of efforts. Once again, there is a tremendous momentum behind Ethereum, and things improve at a rapid pace. Kudos to all the documentation contributors for their work. But by the time we developed our application, the documentation state was clearly not good enough for a new Ethereum developer to start a project.

You can find a few tutorials here and there, but most of the time, copy-pasted code from these tutorials simply doesn’t work.

Here are a few resources worth reading if you want to start developing smart contracts yourself:


After 4 weeks of work by 2 experienced developers, we managed to make our code work in the public Ethereum network with lots of effort. Regressions and compatibility breaks in the Ethereum libraries between Frontier and Homestead versions didn’t help. Check the project source code at marmelab/ZeroDollarHomePage for a detailed understanding of the inner workings. Please forgive the potential bugs in the code, or the inaccuracies in this post - we have a limited experience in the matter. Feel free to send us your corrections in GitHub, or in the comments.

We didn’t enjoy the party. Finding our way across bad documentation and young libraries isn’t exactly our cup of tea. Fighting to implement simple features (like string manipulation) with a half-baked language isn’t fun either. Realizing that, despite years of programming experience in many scripting languages, we are not able to write a simple solidity contract is frustrating. Most importantly, the youth of the Ethereum ecosystem makes it completely impossible to forecast the time to implement a simple feature. Since time is money, it’s currently impossible to determine how much it will cost to develop a Decentralized App.

In time and resources, ZeroDollarHomepage represents a development cost of more than €20,000 - even if it’s a very simple system. As compared to the tools we use in other projects (Node.js, Koa, React.js, PostgreSQL, etc.), developing on the blockchain is very expensive. It’s a great disappointment for the dev team, and a strong signal that the ecosystem isn’t ready yet.

Is this bad experience sufficient to make up our mind about the blockchain? How come many startups showcase their blockchain services as successful innovations? What’s the real cost of building a DApp? Read the last post in this series to see what we really think about the blockchain phenomenon.