Designing for online and distributed environments | Management Optional Notes for UPSC PDF Download

Introduction

Technological innovation has modernized human life, with computers and networks increasingly capable of supporting distributed collaborative multimedia applications.

Process of Distributed and Internet Systems

• The process closely resembles the design of single-location systems.
• However, in the case of multi-location deployment, various design considerations come into play.
• The increased number of components introduces a higher potential for failure.
• Key concerns in these systems include guaranteeing reliability, availability, survivability, and performance.

Distributed and Internet Systems

Designing Distributed System

• A distributed system is an application that employs a set of protocols to coordinate the activities of multiple processes across a network.
• These processes collaborate to execute a singular or a small set of interrelated tasks.
• Building a distributed system offers several advantages, including the capacity to connect remote users with remote resources in an open and scalable manner.
• The collective capabilities of distributed components allow for the creation of a significantly larger and more powerful system than the combinations of individual standalone systems.

A distributed system must have the following characteristics

• Fault-Tolerant: Capable of recovering from component failures without executing incorrect actions.
• Highly Available: Able to restore operations, allowing it to continue providing services even when certain components have failed.
• Recoverable: Components that have failed can restart and rejoin the system once the cause of the failure has been addressed.
• Consistent: Can coordinate actions among multiple components, particularly in the presence of concurrency and failure, enabling the distributed system to function like a non-distributed system.
• Scalable: Can operate correctly as some aspect of the system is scaled to a larger size, whether it's the network, the number of users, servers, or overall system load.
• Predictable Performance: Possesses the ability to deliver the desired responsiveness in a timely manner.
• Secure: Authenticates access to data and services, ensuring a secure environment (Birman, Kenneth, 2005).

In distributed systems design, effectively managing failures is a crucial task. Failures fall into two main categories: hardware and software.

• Hardware Failures:
  • Historically significant until the 1980s, with improvements in internal hardware reliability due to factors like decreased heat production, power consumption, smaller circuits, fewer off-chip connections, and enhanced manufacturing systems.
• Software Failures:
  • Remain a major concern in distributed systems despite rigorous testing, accounting for a substantial portion of unplanned downtime (estimated at 25-35%).
  • Residual bugs in mature systems are often categorized into two main types (Gray, J. and Reuter, A., 1993).

Types of failures that can occur in a distributed system

• Halting Failures: A component abruptly stops without any prior notification. Detection often relies on timeouts, as the tester may not receive any indication of the failure. For example, a user's computer freezing is considered a halting failure.
• Fail-Stop: Similar to halting failures but includes some form of notification to other components before stopping. An example is a network file server informing its clients about an impending shutdown.
• Omission Failures: Occur when a component fails to send/receive messages, typically due to insufficient buffering space. Messages may be discarded without notifying the sender or receiver, especially during router overloads.
• Network Failures: Result from a break in a network link.
• Network Partition Failure: Involves the fragmentation of a network into disjoint sub-networks, where messages can be sent within each sub-network but are lost between them. This can occur due to network failures.
• Timing Failures: Violate temporal properties of the system, such as unsynchronized clocks on different computers used to coordinate processes or delayed messages beyond a defined threshold period.
• Byzantine Failures: Encompass various faulty behaviors, including data corruption, loss, or failures caused by malicious programs (Birman, Kenneth, 2005).
• Heisenbug: A bug that appears to vanish or alter its characteristics when observed or researched. An example is a bug that occurs in a release-mode compile but not in debug mode.
• Bohrbug: A bug that does not disappear or change its characteristics during research. Bohrbugs manifest reliably under well-defined conditions.

Heisenbugs are more prevalent in distributed systems than in local systems due to the complexity of obtaining a coherent and comprehensive view of synchronized process interactions for programmers.

Distributed systems use

• LAN-based file server architecture.
• Client/server architecture.

Designing system for LAN

• Creating a system for a Local Area Network (LAN) involves the configuration of cabling, hardware, and software to link workstations, computers, and file servers within a limited geographical area, usually a single building or campus.
• The file server functions as a device overseeing file operations and is accessible to every client PC connected to the LAN.

File server model:

Drawbacks of file server model

• There is excessive data movement. Entire data table must be transferred, instead of individual records.
• It needs for powerful client workstations. Each client workstation must devote memory to a full DBMS.
• There is decentralized data control. Record concurrency control, recovery, and security are complicated.

Designing Systems for a Client/Server Architecture

Client/server architecture in a LAN-based computing environment involves a central database server or engine handling all database commands received from client workstations. Application programs on each client primarily focus on user interface functions.

File Server vs. Client/Server

Designing Systems for a Client/Server Architecture

• Application processing is split between the client and server.
• The client oversees the user interface.
• The database server manages data storage and query processing.
• The database engine, located on the server, constitutes the back-end of the client/server database system, offering database processing and shared access functions.
• The client, serving as the front-end, provides the user interface and functions for data manipulation.
• Application Program Interface (API) refers to standardized software building blocks utilized to ensure common system capabilities, such as user interfaces and printing, facilitating standardized data exchange between clients and servers.

Client/Server Advantages

• Leverages benefits of microcomputer technology
• Processing performed close to data source
  • Improves response time
  • Reduces network traffic
• Facilitates use of GUIs
• Encourages acceptance of open systems

Differences between client server and file server

• Designing distributed systems is a complex task. In client-server applications, the server provides services like processing database queries or delivering real-time stock prices. The client utilizes these services, presenting query results or suggesting stock purchases to users. Communication between the client and server must be consistent, ensuring no data is lost, and it reaches the client in the order sent by the server.
• Distributed systems consist of various servers. For example, file servers manage disk storage units housing file systems, while database servers store and offer databases to clients. Network name servers map symbolic names or service descriptions to values like IP addresses and port numbers for processes providing the service.
• In distributed systems, multiple servers of a specific type, such as file or network name servers, may exist. The term "service" denotes a set of servers of a particular type. A binding occurs when a process needing to access a service becomes associated with a specific server offering that service. Various binding policies determine server selection, such as locality or load balance.
• Distributed services may use data replication, where multiple data copies facilitate local access or enhance availability after a server process crash. Caching, a common practice, involves a process maintaining a local copy of data for quick access. Caching is similar to replication but may result in stale data. Therefore, a policy for validating cached data might be necessary. Communication reliability is crucial between clients and servers.
• Examples of distributed systems include the web, email, DNS, peer-to-peer systems (file sharing, CDNs, cycle sharing), distributed file systems (NFS, ...), sensor networks, and Akamai CDN.

Designing Internet Systems

The development of internet systems is an area where designers play a crucial role in optimizing web coding and ensuring website security. This encompasses the creation, planning, maintenance, and modeling of computer systems based on the internet.

The process of online system design involves several key aspects:

• Standards:
  • Design simplicity is facilitated by adhering to standards.
  • Naming is governed by BIND.
  • Translation follows the Hypertext Transfer Protocol (HTTP).
  • Formatting is accomplished using Hypertext Markup Language (HTML).
• Separation of Content and Display:
  • HTML, with its tag-oriented format, has limitations.
  • Extensible Markup Language (XML) has been introduced to separate content from display, providing the ability to create custom languages.
• Future Evolution:
  • There is a shift from desktop PCs to thin clients, where most processing and data storage occur on the server.
• Aids to Site Consistency:
  • Cascading Style Sheets (CSS) are utilized to provide a set of style rules instructing web browsers on document presentation.
  • Extensible Style Language (XSL) is a specification for separating style from content when generating HTML documents.
• Design Issues Related to Site Management: Customer Loyalty and Trustworthiness are conveyed through:
  • Design quality
  • Up-front disclosure
  • Comprehensive, correct, and current content
  • Connectivity to the rest of the web
  • Data security
  • Personalization
• Web pages are designed to endure over time, considering customer bookmarks, links from other sites, search engine referrals, and the added value of old content.
• System Security is emphasized, recognizing that effective design stems from understanding how a system fits within the organizational framework, as established in technical studies.