|
GDPR Virtual Microservices Architecture
Introduction Searching the internet for Microservices would result in a total disarray of definitions, images and architects which do not do any justice to Microservices. What is Microservices? Microservices architecture, or simply microservices, is a distinctive method of developing software systems that tries to focus on building single-function modules with well-defined interfaces and operations. Our definition of Microservices is: Simply put, Microservices is "Divide and Conquer." The goal here is to create a system of services where each service addresses the following: performance, security, scalability, flexibility, transparency, availability, refactoring and continuous delivery. It also needs to address interfaces with existing-new services, technologies, data types, DevOps, management, development and testing. The following is our basic structure: 1. Outside World or Browser Side: web, cloud, mobile and other technologies 2. Web Tier: web servers, proxy servers and security parameters 3. Application or Business Tier: business unites, business objects, services, engines and supporting services 4. Database Tiers: database servers, storages and distributed data system 5. Web services: internal and external interfaces and communication 6. DevOps Services: development and infrastructure communication and support The following two images are our Virtual Microservices architecture. 
GDPR Virtual Microservices Cluster Architecture 
GDPR Virtual Microservices Business Tier (Application) Architecture Brief Explanations of Our Virtual Microservices Architecture Components and Flow: Mobile, Web and Cloud Clients: Mobile, Web and Cloud Clients are the users of the system, where they can access the system using cell phone-mobile, the internet or they are cloud clients. Cloud clients may have the access and the control of some of the tiers including the databases based on the cloud governance and business agreements. User Interface (UI): There are a lot of old and new technologies for the Frontend, and flexibility and reusability are key ingredients. The following are what we had used in other projects: 1. JSON 2. jQuery 3. JSP 4. JSP Session - Session and Security 5. JavaScript 6. JS 7. CSS 8. HTML 9. HTML5 10. AJAX 11. Include Files Mobile, Machine Learning and Other Technologies: Our mobile development experience is limited to developing Android-Java Apps. As for other technologies such as Machine learning which focuses on the development of computer programs that can access data and use it to learn for themselves. We believe in building intelligent system. Conceptually, we are turning decision-making into mathematical formulas that use business rules values to choose a course of action. Intelligent System: Intelligence is the ability to make decision and learn as you go. Therefore, we need to combine data and decision-making in one object which we call Intelligent Data Access Object (IDAO). Plus we need to insure that we are doing the right processes in making IDAO. Therefore IDAO must be tested and pass the tests. Firewalls: • A software firewall is a program that is installed on the computer with the Internet connection. • Hardware firewalls are used more in companies and large corporations. • These devices are usually placed between the router and the Internet connection. Proxy Server(s): A proxy server is a server that sits between a client application such as a Web browser, and a real server. It intercepts all requests to the real server to see if it can fulfill the requests itself. We recommend that we build a number of servlets as proxy servers plus they would perform the first line of web security. They would handle rerouting of clients requests. Proxy servers help conceal the internal structure and applications. Servlet Verse Restful Web Service (REST): Servlets are Java specific but RESTful web services are not. Servlets are API but RESTful is not. RESTful web service can use Servlets as their implementation but vice versa is not true. Servlets can run in Servlet container only, but RESTful services can run in web container as well. Servlets, Java Servlet Pages (JSP), JavaScript, JS and sessions provide dynamic and secure browser-web server exchange. JSP, JavaScript, JS and sessions would help move code to the browser-side and save the web servers the load of calls or requests. Security must be addressed when developing such browser-side code. REST allows a greater variety of data formats. Coupled with JSON (which typically works better with data and offers faster parsing), REST is generally considered easier to work with. Thanks to JSON, REST offers better support for browser clients with possible formats, such as XML and JSON. We are prejudice toward Servlets and are also open to use REST as it is the standard for Spring frameworks. Authentication: Authentication is the process of recognizing a user's identity. The credentials provided are compared to those on existing in the system. The proxy servers would run the authentication software. Authentication Services: We recommend developing Authentication code as a service, where it would use encryption as an added security measure against internal hackers. We use Network Attached Storage (NAS) instead of database access to help speed access processes. Network Attached Storage (NAS): NAS is an inexpensive network storage, it is a complete system designed for heavy network systems, which may be processing millions of transactions per minute. NAS provides a widely supported storage system for any organization requiring a reliable network system. 
Can NAS be used as an Authentication storage? NAS is not necessarily connected to the Internet. Security from a security standpoint, NAS devices either provide file system security capabilities of their own, or an alternative to databases to be used for authentication purposes. Disk Utilities (History + Backup + Recovery + Status): We recommend developing Disk Utilities software as services for timestamp tracking of transactions, memory dumps, saving Java objects to the disk, backups, point of recovery, or saving a running application status. NAS is an ideal for such services. Business Units: A logical element or segment of a company (such as accounting, production, marketing) representing a specific business function, and a definite place on the organizational chart, under the domain of a manager. Also called department, division, or a functional area. A business unit is established when a company wants to designate different branding, whether by a product or by target a buyer. Remote Business Unit: Business unit located in a different or remote geographical area. An organization may have a main office, as well as remote offices in another city, country or continent. Testing Business Unit: Testing Business Unit is a running Business Unit for testing purpose. It is running in real time which is an added flexibility to the virtual structure. Virtual Cluster of Business Units: Each of the Business Unit, Disk Utilities software, Authentication Services , Proxy Server or Firewall can all be created as an independent virtual server which runs independently. All these independent servers can also be grouped and run within a virtual server which we call a Virtual Cluster. A number of virtual clusters can be grouped and run within another independent as a Virtual Network. The limit on the number of such virtual servers is dependent on bare-metal server and its capabilities and supporting utilities. Virtual Testing Servers: Virtualization gives the ability to create or delete any number of virtual servers. The types of software running within these virtual services are not limited. Such flexibility gives the option of create any number of testing virtual servers and run any software. Therefore, we can create a testing virtual server as a Business Unit or even an entire Database Container with other virtual components. Virtual Components and Virtual Containers: Virtual Component is a virtual server which runs within it a single software. Virtual Container is a virtual server has a number of Virtual Components and other Virtual Containers. Therefore, Database (DB) Container #1, 2 and 3 are Virtual Containers which have a number of Virtual Components and other Virtual Containers. With the same virtual principles for virtual clusters, we can create each of DB Authentication, Exchange Buffer, DB Adapters, Logging, Audit Trail and Database#X as a virtual server. They all are also Virtual Components Testing-DB Container: It is a Virtual Container which is also a virtual server. It has a number of Virtual Components created for testing purposes in real time. Testing Virtual Containers and components can be used for testing, recovery, rollbacks and production releases. Virtual Servers and IP Addresses: Using IP addresses in redirecting traffic can be used for testing, recovery, rollbacks, production and post production. For example, after the completion of testing a Virtual Container (new production), the production IP address can be redirected to the tested Virtual Container. Now the new virtual testing server would become the virtual production server. Rollback can also be easily done with redirecting the production IP address back to previous production server. Database Services (Level #5) - Virtual Database Servers: The level is a way of numbering the levels of each the services API and their sequence in calls. DB Authentication: This Authentication is for controlling the database access. It also uses Authentication Services and NAS. Exchange Buffer: The buffer is designed to hold upcoming information to provide the smoothest possible consumption experience. The buffer is designed to account for inconsistent data provision speed changes and temporary service loss. Additionally, buffering before starting a stream can provide the stream at a higher-quality than in real-time. Exchange Buffer would be built to speed the database requests and minimize the requests issues. Database Adapters (DB): DB Adapters are Java objects (Factories) for databases access. They also perform database security such as SQL injections and unauthorized database access calls. DB Adapters can be developed to synchronize with Data Exchange services to handle data format and conversions for Microservices. Logging: Experience indicates that logging was an important component of the development cycle. It offers several advantages. It provides precise context about a run of the application. Once inserted into the code, the generation of logging output requires no human intervention. Moreover, log output can be saved in persistent medium to be studied at a later time. In addition to its use in the development cycle, a sufficiently rich logging package can also be viewed as an auditing tool. An application log is a file of events that are logged by a software application. It contains errors, informational events and warnings. The format and content of an application log are determined by the developer of the software program, rather than the OS. See our: • Tracking (Logging-Exceptions-Errors Handlers) On/Off Logging: The ability to control On/Off logging is critical to system performance in peak times. The system architect should include such control. Sadly some developing languages have limitations and the architect would require to find the best practice in such implementation. Log as an Added Security: Log source code and files can be architected to implement security measures in real time. Also log files can be parsed and evaluated for other security flags. Audit Trail: An audit trail (also called audit log) is a security-relevant chronological record, set of records, and/or destination and source of records that provide documentary evidence of the sequence of activities that have affected at any time a specific operation, procedure, or event. Audit records typically result from activities such as financial transactions, scientific research and health care data transactions, or communications by individual people, systems, accounts, or other entities. An audit trail (also called audit log) is a security-relevant chronological record, set of records, and/or destination and source of records that provide documentary evidence of the sequence of activities that have affected at any time a specific operation, procedure, or event. Database Visualizer: Looking at Google techniques for speeding search engines where indexing is used to find data faster. Search engine indexing collects, parses, and stores data to facilitate fast and accurate information retrieval. Our Database Visualizer creates a virtual copy of the existing database and performs the followings: • Parses the existing data in the database based on the business • Implement the index concept • Use statistics for often used tables and files to build our virtual database. • We build a step by step processes to automate the recreation of the virtual database • We use disk space for backup and recovery • Eliminate data duplication and redundancies • Simplify and speed database processes such as database access, cashing, store, edit, and search • Built-in documentation • Create internal and external interfaces • Built-in security features Database#X: In a nutshell, we are parsing the existing database(s) and divide its contents into logical-virtual databases. We give each virtual database an ID number or a name. Our Database Visualizer would built each virtual database based on the Business Unit, Business Objects and Mircoservices structure. Such a virtual database must be a singleton or only one copy running otherwise would be end up in conflicts and inventories issues. The number of these virtual databases is limited and must reflect the main database(s) contents. Management: see our PMP - http://gdprarchitects.com/ManagementPlansPage.html Logging Management: This is similar to logging where Java code would write to management files similar to log files, but only critical code or objects are logged. These files must incorporated in other tools such as MS Project or any management tools. We are planning on creating a GUI interface tools which would parse the Management Log file and translates them into management tracking and suggestion or guidelines . Our approach to management is done as follows: • Project Management Planner (PMP) processes - see our PMP • Manager Free-Format documentation and templates: manager choose his own tools and templates Documentation: Our approach to documentation is to build documentation levels starting with the development code and cover each development, interfaces and executions. Use templates and processes (see our Project Management Planner (PMP) Framework Architect) to structure project documentation. We also built a dedicated virtual server for documentation with access privileges. Content Distribution Network (CDN): Content Distribution Network (CDN) is a large distributed system of servers deployed in multiple data centers across the Internet. The goal of a CDN is to serve content to end-users with high availability and high performance GoldenGate: Oracle GoldenGate is product used to replicate and integrate transactional data with sub-second speed among a variety of enterprise systems. Business Domain (Tier) - - - Virtual Servers + DevOps Support: We have clusters of Business Unit and each Business Unit is a virtual container (virtual server). Coming soon |
|---|
