Explorer DAO - Road to a first-class self-sustainable Explorer for Harmony

This proposal is moved from Explorer DAO topic.

Explorer DAO - Road to first-class self-sustainable Explorer for Harmony

The explorer is one of the most important tools of any Blockchain in all aspects. Beyond the presentation of data of what is happening in the chain, its objective is to serve as a source of truth and reliability for the entire community.

Especially for Harmony, a protocol based on EVM, with a block time as low as 2 seconds and sharding, the proper functioning of the Explorer is more important than any other chain. Blockchains with EVM allow the development of a large number of protocols, applications focused on Defi as well as Games that need to have and show their result as quickly, correctly and understandably as possible. This is a key factor for decision-making, correct visualization and peace of mind for the vast part of the community that does not have nodes or does not need to have very advanced knowledge about them.

In addition, for a blockchain that has as its goal and vision the consensus for 10Bn people, it requires having a readable and understandable browser interface for 10Bn people, facilitating access and clarity in what is happening in the blockchain.

Recently it seems that many members of the community have been having problems with this app as expressed in this post Etherscan version for Harmony One last December 2021 and in the following Reddit posts as of that date:

In any case, at Peersyst we believe that it is vitally important to achieve the following points:

  1. Reliability and security of the information presented.
  2. Speed ​​in indexing , displaying and updating everything that happens in the Harmony chain.
  3. Usability and clarity to display the information presented.

With this vision in mind, the Peersyst team has put a plan for the Harmony DAO Explorer that will enable the following goals:

  • Short term : Availability, reliability and correct display of the information of the Harmony chain.
  • Mid term : Speed ​​and accessibility of the information indexed through API, as well as usability and scalability of the Explorer in both backend and frontend.
  • Long Term : Advanced functionalities that allow both traders, validators, developers and contributors to measure and acquire all the information necessary to develop their activity in Harmony. Sustainability of the project, maintenance and constant growth.

1. State of the art

The first and most important task when establishing a successful plan for the Explorer is to ground the current state of the art from an engineering point of view and locate the components, elements and requirements that the product has. The DAO has been asked to focus in three main directions:

  1. Infrastructure : It consists of taking ownership of the project and maintaining the necessary infrastructure to achieve 99.9% up-time in addition to the necessary scalability required by both the project and the blockchain.
  2. Product : Define a product roadmap and features that help not only improve but expand the scope and utility of the explorer data to achieve a world-class product for EVM.
  3. Self-Sustainability : Finally, establish a medium / long-term plan that allows financially sustaining the costs of infrastructure, maintenance and product development.

From Peersyst we have made an exhaustive analysis of the current situation to be clear about this and to date we have been able to draw the following conclusions:

In general, the project does not have documentation, something that is essential for a product of this size and that makes it easier for the DAO and other future actors to be able to participate and manage the product without serious acquisition costs. In addition, the Explorer is currently very unstable and on several occasions it is not precise in the information that is displayed.

1. Infrastructure: The Explorer uses the AWS infrastructure, which is of great quality and without a doubt the best option for the product’s requirements, although the costs are higher than in other providers, we believe that its quality is worth it. At the moment it does not have a CI/CD system that allows rapid deployment, bug fixes and unit and end-to-end testing of the platform, which is essential. Nor does it have an adequate system of logs, monitoring and traceability of incidents that is essential for the product.

2. Product: we have separated the exploration into the two main parts of the application:

  • Frontend: At the frontend level we can conclude that the practices of the code are good, the product is solid and functional. It has a UX/UI similar to that of Etherscan that we consider very optimal and familiar to many users. However it lacks tests and the UX/UI needs to be improved to be scalable to more future features. Filtering and display of many types of transactions can also be improved.
  • Backend: At the backend level, the situation is a bit different. We believe that from a technological point of view there can be improved code practices that will result in greater scalability and quality of the product. It also has no tests and the performance also has a good margin for optimization.

3. Self-sustainability: The current product is not competitive in a global market and we believe that it is currently difficult to monetize through licensing. In addition, being open-source makes this task even more difficult. It also does not have an API that can be monetized like Etherscan. The product has a good number of users and visits and we believe that with the proper optimizations it will be able to scale much more.

2. Proposed Solution

From Peersyst we propose to carry out the following plan to improve its current situation, to the point of taking it to a much more competitive level and turning it into a world-class product, which we can guarantee its execution.

In general, and as a general work guide, we are going to establish the following points:

  1. Bring together all the microservices in a monorepo to improve the organization and control over the project.
  2. Document the project to make it easier to acquire, participate and contribute and ensure that the ownership is available to the Harmony community.
  3. Stabilization of the product , which requires immediate attention.
  4. Improve performance to optimize its operation.

With a more solid and prepared base, we are going to focus our efforts on the 3 action points that the DAO has set:

1. Infrastructure: TAs an infrastructure, we will continue to use AWS to achieve the highest quality of service to ensure the 99.9% uptime that has been requested. In addition to maintaining this current infrastructure, we are going to make the following improvements:

  • Autoscaling with Lambda or K8’s , which will allow us to scale the Explorer API in number of requests.
  • Implement a CI/CD system with Jenkins that allows us to be agile and secure in the development and deployment of new features and version fixes.
  • Introduce public logs and monitoring dashboards to maintain detailed and absolute control of the product.

2. Product: As in the previous section, the product will scale in its two main components:

  • Frontend: The frontend is in good condition so it is worth keeping the current project. We are going to work with our UX/UI design team to improve the visualization and filtering of transactions to offer the best clarity and experience to the entire community, as well as to guarantee a scalable UX in terms of the number of features that the product can have. Finally, we are going to extend the frontend to support new features as they are implemented. The community will be able to participate in deciding the features that are needed as well as their priority, in such a way that they meet the needs of Harmony. Our team takes a close look at Etherscan features such as gas calculator, NFT tracking, node map and visualization, advanced Harmony network statistics, and mining calculator. All proposals will be welcome in the DAO!
  • Backend: On the backend side, we believe that a refactor is necessary and that it will mean an exponential improvement of the product. To do this, we are going to develop a new backend that is scalable and adapted to the needs of the product. This backend will also use the NestJS framework, which is one of the most popular in the NodeJS ecosystem that allows you to create APIs in a scalable and modular way, as well as TypeORM for connections to the database. This will allow us to have a very powerful and scalable API that we can open to the community and even apply a freemium model in the future for the sustainability of the DAO. We are also going to add a queuing system to add horizontal scalability of the indexer when processing events such as inner transactions or smart contracts among others. We will also add a direct Explorer connection to the database of our internal Harmony nodes to improve performance in data fetching and processing. Finally, both unit testing and e2e will be added to minimize bugs.

3. Self-sustainability: Finally, with a view to the financial sustainability of the project, we are going to focus on several points similar to those used by Etherscan to ensure that we maintain a clean and usable product. The main focus will be the new API, which will have a plan similar to this: Etherscan APIs, which allows us to scale organically with the growth of products and projects in the Harmony community. Also the implementation of B2B deals for product licensing are a valid option in the long term when it is ready. The Peersyst team has been working for years with the “core” team of multiple blockchains and projects and has the infrastructure and experience to execute this type of collaboration. Even so, being open-source for the community, it will be more difficult to compete with private companies. Finally, and if the infrastructure, maintenance and development costs have not been covered, minimal advertising can also be added, mainly focused on Harmony projects in search or on small banners to get extra income and more diversification.

With this clear view of the points to be solved, Peersyst has established a focused plan with different milestones and phases that will aim to solve these points in a parallel and organized manner with the shortest possible time, which is specified in section 5.

3. Team

The Peersyst team has been defined and designed as a high-performance squad capable of developing any project in Blockchain technology with maximum efficiency, quality and speed.

With a Blockchain development experience practically as long as Ethereum years of existence, the Peersyst team has been carefully selected as one of the most expert and complemented teams in the development of this technology.

It has the following roles:

PRODUCT OWNER / REQUIREMENTS ENGINEER - SI ENGINEER
The main mission of the Product Owner is to define the product and prioritize the objectives to be achieved, in order to maximize the value of the product.
He is the representative of all interested persons (stakeholders) and must ensure that the Product Result meets the expected results.

TECH LEAD - TI ENGINEER
The tech lead is responsible for the entire technological development and definition part of the project. It is responsible for defining the most appropriate technologies, the Blockchain protocols to be used, the test and production environments, the platform infrastructure, security standards, backups, migrations and integrations with other systems. The highest level technology profile responsible for the platform and development success.

UX DESIGNER - DESIGNER
Usability is one of the most important challenges posed by Blockchain technology. Giving access to it for all audiences is a challenge that is the job of the UX team to solve. Peersyst incorporates a successful profile in the design of platforms that have achieved millions of users thanks to the usability and ease of its solutions.

UI DESIGNER - DESIGNER
This profile is of vital importance for the success of the projects. Design of easy and attractive interfaces for users makes a big difference between the clarity and understanding of the information. Peersyst incorporates a successful profile in the design of platforms that have achieved millions of users thanks to the usability and ease of its solutions.

BLOCKCHAIN / CRYPTOGRAPHY SPECIALIST - COMPUTATION AND ALGORITHMICS
This profile is in charge of helping the tech lead to define the specific Blockchain solutions and develop the previously defined protocols. Its functionality is to ensure the security of all connections and of the platform at the software level.

SENIOR BACKEND DEVELOPER - COMPUTATION AND ALGORITHMICS
This profile is in charge of developing all parts of the solutions on the server side. Many platforms can use a backend as an engine for logic and allow integrating the frontend with Blockchain modules, user management and administrators.

SENIOR FRONTEND DEVELOPER - SOFTWARE ENGINEER
The developer responsible for designing all the changes in the visual part of the applications and the part of the logic that does not interact with the Blockchain.

FULLSTACK DEVELOPER - SOFTWARE ENGINEER
Finally, the fullstack developer is a flexible and agile resource that is in charge of integrating with third parties, connections between frontend and backend, and providing support in the different areas above.

4. Budgeting

We purpose funding the Explorer DAO with USD 937.020 to develop its activities to achieve the detailed roadmap and start the self-monetization phase .

We would like to ask for USD 468.210 to kick-start activities on the development until complete phase 1, with the launch of Explorer V3. Also we need the budgeting for the AWS cost for this month in order to get the service and secure the infrastructure.

AWS Cost AWS Pricing Calculator

5. Project Milestones

Phase 1: Explorer V2 - Ownership, stabilization and development of the new version of the Explorer (6 months)

Phase 1.1: Ownership and infrastructure (1-2 months)

The objective of the first phase will be to take ownership of the product and the code and to stabilize the project. Basically it will be a phase to start up the entire infrastructure, stabilize the product and attend to the issues that come out. The following tasks will be executed:

  • Issue resolving: Being sure there is no issue older than 1 week.
  • Add monitoring and rescue system
  • Add CI / CD for rapid delivery
  • Add integration and unit testing

Phase 1.2: Building Explorer V3 and Maintenance (3-4 months)

The objective of the second phase, with a product running, maintained and stabilized, will be focused on product growth and quality. The following actions will be carried out:

5. Maintenance of the product to guarantee its operation

  • Issue resolving
  • Monitoring

6. Backend: Implementation of the new API including all the frameworks and technological improvements that have been previously exposed:

  • Development of the new backend using the NestJS framework and TypeORM.
  • Improve the performance connecting the Explorer directly with the Harmony node DBs.
  • Add a queue system to achieve horizontal scalability.

7. Frontend: On the frontend side, efforts will be devoted to designing a product that is more scalable in terms of functionality and improving usability and content display.

  • Maintenance of the product.
  • UX study to increase the inclusion of product functionalities.
  • Design of a UI adapted to the new branding and that facilitates the reading and understanding of data.

Phase 2: Explorer V3 - Launch of the new product and expansion of functionalities (6 months)

Following the conclusion of the previous phase, the Explorer V3 is ready for launch. In this phase we are going to launch the product and we are going to make sure of its stability and correct operation, with a much better performance for the community, with the new public API and with a new UX/UI adapted to its growth.

The objectives of this phase will be to develop new features as well as to stabilize Explorer V3.

1. Maintenance

  • Issue resolving
  • Monitoring

2. Launch Explorer V3

  • Stabilize the Explorer V3
  • Add a production environment setup with autoscaling (Lambda or K8s)
  • Add pipelines for production environment
  • Add monitoring for production and staging environment

3. Design and development of new features (Explorer V4)

  • Feature definition (Gas calculator, NFT tracking, Node map and visualization, Advanced Harmony network statistics, Mining calculator)
  • UX/UI Design for the new screens
  • Development of the new features

Phase 3: Explorer V4 - New features and monetization (X months)

In this phase, the explorer will already be world-class, having not only excellent performance and data functionality, a public and previously tested API, as well as new extra features that may be relevant for multiple projects.

1. Maintenance

  • Issue resolving
  • Monitoring

2. Launch Explorer V4

  • Stabilize the Explorer V4

3. Monetization Strategy

  • API: The monetization strategy will go through establishing certain limitations of requests to the API as well as premium data focused on large platforms and never on the small user.
  • License: Licensing of the Explorer on a B2B basis in other compatible EVM projects.

4. Design and development of new features (Explorer VX)

  • Feature definition (Automatic API payments and invoices, community requests, etc)
  • UX/UI Design for the new screens
  • Development of the new features

6. Timeline & Budget

AWS Cost AWS Pricing Calculator

5 Likes

Thanks for the proposal. I would like to see a clear mile of when can we do the cutover so that we can stop our Harmony internal instance of explorer? This is the very first you must archive as the deadline for the proposal. My understanding is the first phase, correct me if I am wrong.

1 Like

Hi @leo ! As soon as the proposal is approved we can start the migration and manage the Explorer from the DAO.

To go quietly about 15 days from the payout to have time to receive the funds, change the necessary part and contract the first months of infrastructure in AWS, while we migrate and deploy the frontend / backend and we take ownership of the code :+1:t2:

1 Like

Thanks for the update. So I’d make it clear that the explorer will be fully taken over by your team and the DAO one month after the agreement and payment, including the db, backend, frontend, load balancer, deployment and issues. That looks very ambitious but very promising.

Just a note to you that we are currently looking into the cloud db cost, mostly coming from RDS. You may consider to have your own Postgres cluster setup.

1 Like

Hi @leo!

Thank you for your comment and feedback, it is of great value. Indeed, our team has the expertise and capabilities to fully take over the Explorer within a month of the agreement. Of course, we appreciate all the help in the transmission of knowledge and infrastructure that the current Harmony team can provide during this period to facilitate the task.

You are absolutely right @leo, with a Postgress cluster we can probably reduce certain costs and it is something to take into account in the medium term. In any case, regarding the costs of the cloud db, in no case should they be higher than the costs that you currently support, this is the configuration information that we obtained and the most optimal and fastest way to migrate in the first instance.

We remain at your disposal ready for next steps! :+1:t2:

1 Like

After working and validating the following steps to proceed with the DAO Explorer, we propose the following governance model to start with:

Signatories:

  • FP is the CEO & Founder at Peersyst Technology, a leading blockchain development company and an active and experienced contributor in the Harmony ecosystem. With a computer engineering background at UPC, he has 6+ years of experience in Blockchain development, actively contributing in multiple protocols and with 30+ successfully launched projects. The company also holds some of the biggest blockchain communities in Barcelona and has been chosen as a partner in Blockhcain in several governments and recognized multinationals. Ferran has also been a speaker at multiple events and conferences internationally and author of various blockchain trainings both for public and private sector.

  • AC is the CTO & Founder at Peersyst Technology, the leading technology person within the company and the person in charge of defining all the processes, methodologies, technologies and activities of the entire development team of the company within all the different disciplines, protocols and technological projects that Peersyst carries out. Adria has training as a Computer Engineer with a specialty in telecommunications from the UPC.

  • MP is the CFO and the Head of Finance at Peersyst Technology since the start of the company. She has a background in economics from UPF in Barcelona and during her professional experience she has worked as Head of Controlling in multinationals, designing and implementing the entire accounting system of the company on several continents.

  • MC is a designer specialized in Blockchain technology. He currently works in the UX / UI team at Peersyst Technology and is in charge of defining the usability and interface of most Peersyst products. He will be in charge of defining, scaling and preparing the entire infrastructure and visual architecture of the Explorer to facilitate its access and usability while adapting to the new branding and allowing access to the new features that are being implemented.

  • PG is the Director of Business Development Latam Region at Peersyst Technology. Pedro also has a background in systems engineering, and during his professional career he has assumed C level and Council Member roles in widely recognized blockchain protocols such as NEM. He currently leads the entire Peersyst Technology strategy in Latin America where technology is applied with different governments, banks and multinationals.

  • EP holds a Ph.D. and Master of International Affairs. He is an international development expert with +10 years of experience working on projects financed by USAID, USIP, etc. and has worked in the Sahel for numerous years. He has actively collaborated with other organizations in the past, such as the Ethereum Foundation and Polygon, reviewing the African crypto landscape, mentoring local start-ups and supporting them in grant applications.

  • GP is the Blockchain Product & Engineering at Harmony Protocol and the driver of this DAO. Giv has a professional Certificate, Data Science: R and Machine Learning from HarvardX and has experience as an engineer in companies such as AWS. I think the entire Harmony community knows Giv’s criteria and professionalism!

  • JP is Software Engineer at Harmony Protocol and one of the people in charge of Explorer currently. He knows well all the needs and technology both in the frontend and backend and his experience and knowledge are little more than indispensable for the success of the DAO and for the transfer of knowledge!

  • LC is a Computer Scientist & Engineer from Zhejiang University and currently none other than the VP of Engineering at Harmony Protocol. Due to his background, knowledge and involvement in the product and in the protocol, Leo will be a key role in governance, fund management and supervision so that the Explorer has a satisfactory transition and development with the objectives that have been set.

Signatories:
For signatories, we have taken inspiration from other DAO proposals that have been widely accepted and are working satisfactorily such as Harmony Philippines DAO - #16 by laet4x, since this project has a long-term vision and extension, so it is important to find a model that allows regulating actions, direction, functionalities and funds of the DAO while maintaining a certain stability that guarantees certain investments in hardware as well as stability of the roadmap in the medium term.

Harmony Explorer DAO signatories rotate on a 3-month basis. The election process should follow a per quarter basis.

Qualifications to Become Signatories:

  1. Signatory should have past experience in engineering, product design, business development or running a Harmony node.
  2. Fluent in English.

Election 1st Round: Election of the current signatory for Top 6

  1. Re-election of signatories and will be voted by Signatories + Community.
  • Top 1 to Top 6 will be retained.
  • Top 7 to Top 9 will be opened for election.
  1. Top 7 to Top 9 positions will be opened for the community members. The previous signatory can run for re-election.

Schedule for 1st round election:

  1. Registration

  2. Voting

Election 2nd Round: Election of 3 New Signatory for the Top 7 to Top 9 position

  1. Filing of Candidacy
  2. Campaign period
  3. Election Period

Mandates (Extracted from the original DAO request):

  • Operations : Complete ownership of the Explorer stack (front-end/back-end).
  • Maintenance : Resolve inbound issues in a timely manner (no more than 1 week)
  • Product: Process feature requests and implement a compelling product roadmap.