This proposal is moved from Explorer DAO topic.
Explorer DAO - Road to first-class self-sustainable Explorer for Harmony
The explorer is one of the most important tools of any Blockchain in all aspects. Beyond the presentation of data of what is happening in the chain, its objective is to serve as a source of truth and reliability for the entire community.
Especially for Harmony, a protocol based on EVM, with a block time as low as 2 seconds and sharding, the proper functioning of the Explorer is more important than any other chain. Blockchains with EVM allow the development of a large number of protocols, applications focused on Defi as well as Games that need to have and show their result as quickly, correctly and understandably as possible. This is a key factor for decision-making, correct visualization and peace of mind for the vast part of the community that does not have nodes or does not need to have very advanced knowledge about them.
In addition, for a blockchain that has as its goal and vision the consensus for 10Bn people, it requires having a readable and understandable browser interface for 10Bn people, facilitating access and clarity in what is happening in the blockchain.
Recently it seems that many members of the community have been having problems with this app as expressed in this post Etherscan version for Harmony One last December 2021 and in the following Reddit posts as of that date:
In any case, at Peersyst we believe that it is vitally important to achieve the following points:
- Reliability and security of the information presented.
- Speed ​​in indexing , displaying and updating everything that happens in the Harmony chain.
- Usability and clarity to display the information presented.
With this vision in mind, the Peersyst team has put a plan for the Harmony DAO Explorer that will enable the following goals:
- Short term : Availability, reliability and correct display of the information of the Harmony chain.
- Mid term : Speed ​​and accessibility of the information indexed through API, as well as usability and scalability of the Explorer in both backend and frontend.
- Long Term : Advanced functionalities that allow both traders, validators, developers and contributors to measure and acquire all the information necessary to develop their activity in Harmony. Sustainability of the project, maintenance and constant growth.
1. State of the art
The first and most important task when establishing a successful plan for the Explorer is to ground the current state of the art from an engineering point of view and locate the components, elements and requirements that the product has. The DAO has been asked to focus in three main directions:
- Infrastructure : It consists of taking ownership of the project and maintaining the necessary infrastructure to achieve 99.9% up-time in addition to the necessary scalability required by both the project and the blockchain.
- Product : Define a product roadmap and features that help not only improve but expand the scope and utility of the explorer data to achieve a world-class product for EVM.
- Self-Sustainability : Finally, establish a medium / long-term plan that allows financially sustaining the costs of infrastructure, maintenance and product development.
From Peersyst we have made an exhaustive analysis of the current situation to be clear about this and to date we have been able to draw the following conclusions:
In general, the project does not have documentation, something that is essential for a product of this size and that makes it easier for the DAO and other future actors to be able to participate and manage the product without serious acquisition costs. In addition, the Explorer is currently very unstable and on several occasions it is not precise in the information that is displayed.
1. Infrastructure: The Explorer uses the AWS infrastructure, which is of great quality and without a doubt the best option for the product’s requirements, although the costs are higher than in other providers, we believe that its quality is worth it. At the moment it does not have a CI/CD system that allows rapid deployment, bug fixes and unit and end-to-end testing of the platform, which is essential. Nor does it have an adequate system of logs, monitoring and traceability of incidents that is essential for the product.
2. Product: we have separated the exploration into the two main parts of the application:
- Frontend: At the frontend level we can conclude that the practices of the code are good, the product is solid and functional. It has a UX/UI similar to that of Etherscan that we consider very optimal and familiar to many users. However it lacks tests and the UX/UI needs to be improved to be scalable to more future features. Filtering and display of many types of transactions can also be improved.
- Backend: At the backend level, the situation is a bit different. We believe that from a technological point of view there can be improved code practices that will result in greater scalability and quality of the product. It also has no tests and the performance also has a good margin for optimization.
3. Self-sustainability: The current product is not competitive in a global market and we believe that it is currently difficult to monetize through licensing. In addition, being open-source makes this task even more difficult. It also does not have an API that can be monetized like Etherscan. The product has a good number of users and visits and we believe that with the proper optimizations it will be able to scale much more.
2. Proposed Solution
From Peersyst we propose to carry out the following plan to improve its current situation, to the point of taking it to a much more competitive level and turning it into a world-class product, which we can guarantee its execution.
In general, and as a general work guide, we are going to establish the following points:
- Bring together all the microservices in a monorepo to improve the organization and control over the project.
- Document the project to make it easier to acquire, participate and contribute and ensure that the ownership is available to the Harmony community.
- Stabilization of the product , which requires immediate attention.
- Improve performance to optimize its operation.
With a more solid and prepared base, we are going to focus our efforts on the 3 action points that the DAO has set:
1. Infrastructure: TAs an infrastructure, we will continue to use AWS to achieve the highest quality of service to ensure the 99.9% uptime that has been requested. In addition to maintaining this current infrastructure, we are going to make the following improvements:
- Autoscaling with Lambda or K8’s , which will allow us to scale the Explorer API in number of requests.
- Implement a CI/CD system with Jenkins that allows us to be agile and secure in the development and deployment of new features and version fixes.
- Introduce public logs and monitoring dashboards to maintain detailed and absolute control of the product.
2. Product: As in the previous section, the product will scale in its two main components:
- Frontend: The frontend is in good condition so it is worth keeping the current project. We are going to work with our UX/UI design team to improve the visualization and filtering of transactions to offer the best clarity and experience to the entire community, as well as to guarantee a scalable UX in terms of the number of features that the product can have. Finally, we are going to extend the frontend to support new features as they are implemented. The community will be able to participate in deciding the features that are needed as well as their priority, in such a way that they meet the needs of Harmony. Our team takes a close look at Etherscan features such as gas calculator, NFT tracking, node map and visualization, advanced Harmony network statistics, and mining calculator. All proposals will be welcome in the DAO!
- Backend: On the backend side, we believe that a refactor is necessary and that it will mean an exponential improvement of the product. To do this, we are going to develop a new backend that is scalable and adapted to the needs of the product. This backend will also use the NestJS framework, which is one of the most popular in the NodeJS ecosystem that allows you to create APIs in a scalable and modular way, as well as TypeORM for connections to the database. This will allow us to have a very powerful and scalable API that we can open to the community and even apply a freemium model in the future for the sustainability of the DAO. We are also going to add a queuing system to add horizontal scalability of the indexer when processing events such as inner transactions or smart contracts among others. We will also add a direct Explorer connection to the database of our internal Harmony nodes to improve performance in data fetching and processing. Finally, both unit testing and e2e will be added to minimize bugs.
3. Self-sustainability: Finally, with a view to the financial sustainability of the project, we are going to focus on several points similar to those used by Etherscan to ensure that we maintain a clean and usable product. The main focus will be the new API, which will have a plan similar to this: Etherscan APIs, which allows us to scale organically with the growth of products and projects in the Harmony community. Also the implementation of B2B deals for product licensing are a valid option in the long term when it is ready. The Peersyst team has been working for years with the “core” team of multiple blockchains and projects and has the infrastructure and experience to execute this type of collaboration. Even so, being open-source for the community, it will be more difficult to compete with private companies. Finally, and if the infrastructure, maintenance and development costs have not been covered, minimal advertising can also be added, mainly focused on Harmony projects in search or on small banners to get extra income and more diversification.
With this clear view of the points to be solved, Peersyst has established a focused plan with different milestones and phases that will aim to solve these points in a parallel and organized manner with the shortest possible time, which is specified in section 5.
3. Team
The Peersyst team has been defined and designed as a high-performance squad capable of developing any project in Blockchain technology with maximum efficiency, quality and speed.
With a Blockchain development experience practically as long as Ethereum years of existence, the Peersyst team has been carefully selected as one of the most expert and complemented teams in the development of this technology.
It has the following roles:
PRODUCT OWNER / REQUIREMENTS ENGINEER - SI ENGINEER
The main mission of the Product Owner is to define the product and prioritize the objectives to be achieved, in order to maximize the value of the product.
He is the representative of all interested persons (stakeholders) and must ensure that the Product Result meets the expected results.
TECH LEAD - TI ENGINEER
The tech lead is responsible for the entire technological development and definition part of the project. It is responsible for defining the most appropriate technologies, the Blockchain protocols to be used, the test and production environments, the platform infrastructure, security standards, backups, migrations and integrations with other systems. The highest level technology profile responsible for the platform and development success.
UX DESIGNER - DESIGNER
Usability is one of the most important challenges posed by Blockchain technology. Giving access to it for all audiences is a challenge that is the job of the UX team to solve. Peersyst incorporates a successful profile in the design of platforms that have achieved millions of users thanks to the usability and ease of its solutions.
UI DESIGNER - DESIGNER
This profile is of vital importance for the success of the projects. Design of easy and attractive interfaces for users makes a big difference between the clarity and understanding of the information. Peersyst incorporates a successful profile in the design of platforms that have achieved millions of users thanks to the usability and ease of its solutions.
BLOCKCHAIN / CRYPTOGRAPHY SPECIALIST - COMPUTATION AND ALGORITHMICS
This profile is in charge of helping the tech lead to define the specific Blockchain solutions and develop the previously defined protocols. Its functionality is to ensure the security of all connections and of the platform at the software level.
SENIOR BACKEND DEVELOPER - COMPUTATION AND ALGORITHMICS
This profile is in charge of developing all parts of the solutions on the server side. Many platforms can use a backend as an engine for logic and allow integrating the frontend with Blockchain modules, user management and administrators.
SENIOR FRONTEND DEVELOPER - SOFTWARE ENGINEER
The developer responsible for designing all the changes in the visual part of the applications and the part of the logic that does not interact with the Blockchain.
FULLSTACK DEVELOPER - SOFTWARE ENGINEER
Finally, the fullstack developer is a flexible and agile resource that is in charge of integrating with third parties, connections between frontend and backend, and providing support in the different areas above.
4. Budgeting
We purpose funding the Explorer DAO with USD 937.020 to develop its activities to achieve the detailed roadmap and start the self-monetization phase .
We would like to ask for USD 468.210 to kick-start activities on the development until complete phase 1, with the launch of Explorer V3. Also we need the budgeting for the AWS cost for this month in order to get the service and secure the infrastructure.
AWS Cost AWS Pricing Calculator
5. Project Milestones
Phase 1: Explorer V2 - Ownership, stabilization and development of the new version of the Explorer (6 months)
Phase 1.1: Ownership and infrastructure (1-2 months)
The objective of the first phase will be to take ownership of the product and the code and to stabilize the project. Basically it will be a phase to start up the entire infrastructure, stabilize the product and attend to the issues that come out. The following tasks will be executed:
- Issue resolving: Being sure there is no issue older than 1 week.
- Add monitoring and rescue system
- Add CI / CD for rapid delivery
- Add integration and unit testing
Phase 1.2: Building Explorer V3 and Maintenance (3-4 months)
The objective of the second phase, with a product running, maintained and stabilized, will be focused on product growth and quality. The following actions will be carried out:
5. Maintenance of the product to guarantee its operation
- Issue resolving
- Monitoring
6. Backend: Implementation of the new API including all the frameworks and technological improvements that have been previously exposed:
- Development of the new backend using the NestJS framework and TypeORM.
- Improve the performance connecting the Explorer directly with the Harmony node DBs.
- Add a queue system to achieve horizontal scalability.
7. Frontend: On the frontend side, efforts will be devoted to designing a product that is more scalable in terms of functionality and improving usability and content display.
- Maintenance of the product.
- UX study to increase the inclusion of product functionalities.
- Design of a UI adapted to the new branding and that facilitates the reading and understanding of data.
Phase 2: Explorer V3 - Launch of the new product and expansion of functionalities (6 months)
Following the conclusion of the previous phase, the Explorer V3 is ready for launch. In this phase we are going to launch the product and we are going to make sure of its stability and correct operation, with a much better performance for the community, with the new public API and with a new UX/UI adapted to its growth.
The objectives of this phase will be to develop new features as well as to stabilize Explorer V3.
1. Maintenance
- Issue resolving
- Monitoring
2. Launch Explorer V3
- Stabilize the Explorer V3
- Add a production environment setup with autoscaling (Lambda or K8s)
- Add pipelines for production environment
- Add monitoring for production and staging environment
3. Design and development of new features (Explorer V4)
- Feature definition (Gas calculator, NFT tracking, Node map and visualization, Advanced Harmony network statistics, Mining calculator)
- UX/UI Design for the new screens
- Development of the new features
Phase 3: Explorer V4 - New features and monetization (X months)
In this phase, the explorer will already be world-class, having not only excellent performance and data functionality, a public and previously tested API, as well as new extra features that may be relevant for multiple projects.
1. Maintenance
- Issue resolving
- Monitoring
2. Launch Explorer V4
- Stabilize the Explorer V4
3. Monetization Strategy
- API: The monetization strategy will go through establishing certain limitations of requests to the API as well as premium data focused on large platforms and never on the small user.
- License: Licensing of the Explorer on a B2B basis in other compatible EVM projects.
4. Design and development of new features (Explorer VX)
- Feature definition (Automatic API payments and invoices, community requests, etc)
- UX/UI Design for the new screens
- Development of the new features
6. Timeline & Budget
AWS Cost AWS Pricing Calculator
