web-project-life-cycle.com Open in urlscan Pro
194.102.33.91  Public Scan

URL: https://web-project-life-cycle.com/
Submission: On November 21 via api from US — Scanned from US

Form analysis 0 forms found in the DOM

Text Content

Web Project Life Cycle

by Marius Daniel Ciorecan

Contents

1       Chapter One - Project Life Cycle. 4

1.1        Main development stages. 4

1.2        Development stages when developing with external implementation
Supplier 5

1.3        Technical implementation flow.. 7

1.4        Project ecosystem.. 9

1.4.1         DevOps. 10

1.4.2         Environments. 14

1.4.3         Planning & tracking tools and practices. 14

1.4.4         Testing. 15

1.4.5         Project methodology and best practices. 15

1.4.6         Maintenance & support 15

2       Chapter Two - Web architecture. 16

2.1        Generic Client Server architecture. 16

2.2        Basic Web architecture. 18

2.3        High load Web architecture. 19

2.3.1         Caching. 20

2.3.2         Scaling. 21

2.3.3         Load balancing. 21

2.4        Authentication & Authorization. 22

2.4.1         Web platform as Service Provider and Identity Provider 22

2.4.2         Web Platform as Service Provider and separate Identity Provider 23

3       Chapter Three – Server details. 25

3.1        Connection and communication. 26

3.1.1         OSI Model 26

3.1.2         Network sockets. 26

3.1.3         Default ports. 27

3.1.4         Web servers. 28

3.2        Processing. 29

3.2.1         Using APIs for Client – Server integration. 29

3.2.2         Internal API vs Public API 33

3.3        Persistence. 34

3.3.1         Object Relational Mapping (ORM) 35

3.3.2         Relational vs NoSQL database. 36

3.3.3         Database programming. 36

4       Chapter Four - Client details. 38

4.1        Web clients. 39

4.1.1         Web sites. 39

4.1.2         Web platforms / applications. 43

4.1.3         Visitor access to a web site (DNS) 47

5       Chapter Five – Best practices and standards. 49

5.1        Logging. 49

5.2        Error handling. 51

5.2.1         Web error handling. 51

5.3        Code style and formatting. 52

5.4        Code analysis. 53

5.5        Code reviews. 53

6       Web development common features. 54

6.1        Cookies. 54

6.2        Browser storage. 55

6.3        Caching. 55

6.4        Authentication and authorization. 55

7       Testing. 58

7.1        Types of testing. 58

7.2        Manual vs Automated testing (Web) 59

 

 

 


1     CHAPTER ONE - PROJECT LIFE CYCLE


1.1   MAIN DEVELOPMENT STAGES

Standard implementation of a Software project (Web or non-Web) consists on the
following stages:

 

1.    Research and Analysis

·         Project need identified by Management/Sales/Business Development

·         High level requirements are put in place. Roles involved: Management,
Project Managers, Business Analysts

2.    Planning

·         Team planning: allocate team & roles

·         Timeline & duration

·         Roles involved: Management, Project Managers, Business Analysts

3.    Design

·         Use cases are put in place. Roles involved: Project Managers,
Business, Technical Architects

·         Designs implementation based on use cases. Roles involved: same as
above + Creative

4.    Implementation

·         Having all the requirements, use cases and designs defined, the
project technical implementation can start.

·         Roles involved: Project Managers, Product Owner, Development, Testing

5.    Testing and Integration

·         The project deliverables are being tested and validated

6.    Deploy and Maintain

·         Project is deployed to live environment

·         The project will usually be maintained and/or extended depending on
various factors

 

Even though described separately, the steps are highly connected and also one or
successions of steps are repeated in multiple iterations. For example:

-       Use cases and designs are built in multiple rounds of reviews

-       Testing is happening through most the implementation and deploy duration


1.2   DEVELOPMENT STAGES WHEN DEVELOPING WITH EXTERNAL IMPLEMENTATION SUPPLIER

The flow is based on the case when the Supplied is the IT company providing the
implementation services. Still, it is pretty similar when the case is addressing
a product development (Client and Supplied in this are the same entity).

 

Below is an example of such an interaction, but in practice the interaction
differs for each case, depending on a LOT of factors:

-       History between Client and Supplier

-       Technology stack used: sometimes the Client has specific platforms and
tools licensed and approved, other times it will rely on Supplier
recommendation.

-       The type of project implemented (Time and Materials, Fixed Price, etc),
based on that some of the roles will be either only on the Client Side, either
only on the Supplier Side, or shared. As an example, for a Time and Materials
project, usually the project management will be done on the Client side and it
might just use the development team of the Supplier as human resources on the
project, where for a Fixed Price project type, project management could be
shared (each party having its own project manager), either completely on the
Supplier side.




1.3   TECHNICAL IMPLEMENTATION FLOW

 

The flow below exemplifies a generic implementation: implement a
component/phase/sprint etc, validate it, deploy it and move to the next one.
Depending on the methodology, the details will differ. For example:

-       On a Waterfall approach:

o   Large components will be divided onto implementation phases.

o   Each phase will be implemented in sequential order:

§  Requirements phase (sometimes split into an additional analysis phase)

§  System design phase

§  Implementation phase

§  Testing phase

§  Deployment phase

§  Maintenance phase

-       On an Agile approach:

o   The development is divided into smaller and iterative steps, named sprints.

o   A sprint will last usually between 2 – 4 weeks.

o   Each sprint goes through the following phases:

§  First, the product owner organizes the product backlog. The product backlog
is a list of every task that may be worked on during the sprint. This
information is usually stored in a project management tool.

§  Before the sprint, the entire project team participates in sprint planning to
identify the best tasks to work on during the two-week period.

§  During the sprint, Agile teams meet frequently to discuss blockers and action
items.

§  Once the sprint is over, team members get together to run a sprint
retrospective and identify what went well and what could have been better.

 

 




1.4   PROJECT ECOSYSTEM

 




1.4.1  DEVOPS

DevOps is a set of practices that combines software development (Dev) and IT
operations (Ops). Will cover the following steps from the development and
delivery:

 

·         Coding – code development and review, source code management tools,
code merging.

·         Building – continuous integration tools, build status.

·         Testing – continuous testing tools that provide quick and timely
feedback on business risks.

·         Packaging – artifact repository, application pre-deployment staging.

·         Releasing – change management, release approvals, release automation.

·         Configuring – infrastructure configuration and management,
infrastructure as code tools.

·         Monitoring – applications performance monitoring, end-user experience.

1.4.1.1   SOURCE CODE REPOSITORY

Code versioning tools. Store/share code, track code history, track releases and
tags.

 

Samples:

 

·         Bitbucket git server



 

·         SourceTree git client



 

Most used tools:

·         git. Server platforms based on git: GitHub, Bitbucket. Client tools:
SourceTree, TortoiseGit, etc

·         Microsoft Team Foundation Server (TFS)

·         svn

 

Common practices:

·         Different branches to work on the code: master/release branches,
branches for main development, branches for different modules, etc

·         Approval workflows: usually the code submitted by a developer will go
through a review process before is added to the main branch.

1.4.1.2   CONTINUOUS INTEGRATION

Build/deploy (and more) automation process platform.

Most used tools:

·         Jenkins/Hudson

·         TFS

 

Samples:

·         Builds configuration

 




1.4.2  ENVIRONMENTS

We define an environment as the collection of tools (hardware & software) to
host all the components of a project. Usually will contain:

·          The machine(s) that host the project deliverables (cloud, VM, bare
metal) with the required Operating System. For a Web Project for example we
could have:

o   One ore multiple Windows OS machines running IIS Web Server / Apache / nginx
etc (usually is IIS)

o   One ore multiple Linux/Unix OS machines running Apache / nginx etc

·         The machines(s) that host the project database. Usually, a project
will use a database for persistence.

 

 

Usually on a project you will find the environments below (some of them are
optional though):

·         Local environment – running on each developer machine

·         Development environment – (Supplier side) available to all developers
& testers, always runs the latest changes. Used usually by developers.

·         Testing environment – (Supplier side) – available to all developers &
testers. Will run the current release / sprint etc. Primarily used by Supplier
QA.

·         Staging environment – (Client or Supplier side) – available to
supplier and client. Will run the official releases signed by the Supplier.
Primarily used by Client QA.

·         Preview environment – (Client side) – final step before live
deployment, usually connected to the Live / Production database.

·         Live environment

 


1.4.3  PLANNING & TRACKING TOOLS AND PRACTICES

 

·         Planning

o   Planning starts before project actual start

o   The initial project team is selected and planned (Excel or web tools)

o   The project team is usually flexible, people are added/removed from the
project as the project progresses.

 

·         Tracking / Tasks Management

o   Using web tools to collaborate and track the team progress

 


1.4.4  TESTING

 

Next to Development, Testing is one the most important components of the
Implementation process. Depending on the project, there are various testing
methodologies that can be applied (one or more):

·         Manual testing – testing done by QA teams based on the Requirements &
Use cases

·         Automated testing – for Web implementation there are platforms where
you can program automated testing from the browser (Selenium, Cypress, etc)

·         Unit testing – code testing

·         Smoke testing – usually applies to the Live environment, a smaller
group of manual / automated testing to make sure the project functionality
hasn’t been break by the latest deploy/


1.4.5  PROJECT METHODOLOGY AND BEST PRACTICES

 

·         Also, at the start of the project is decided the implementation
methodology: Waterfall, Agile, Scrum, Kanban, or custom mix.

·         Code reviews, code formatting, code analysis tools

·         Logging

·         Exception handling


1.4.6  MAINTENANCE & SUPPORT

 

Usually after the project is completed or when parts of the project have been
deployed live. Will consists of:

·         New features

·         Change requests

 

 


2     CHAPTER TWO - WEB ARCHITECTURE

 


2.1   GENERIC CLIENT SERVER ARCHITECTURE



 

Will consist of a Server application and one or more Client applications,
exchanging data through a network. Based on the OSI layers below, communication
can be implemented at various OSI layers. Example:

 

·         Industrial client apps exchanging data with a Server through TCP (OSI
4) protocol. The data will be usually raw bytes information encoded on agreed
formats: scanners on production lines transmitting continuous information to a
Server

·         Media stream providers using UDP (OSI 4) for video transmissions

·         Web platforms using HTTP protocol (OSI 7)

·         File copies through FTP (OSI 7)

·         Email client apps (mobile, desktop) communication with email server
through SMTP/POP3 (OSI 7)

 

OSI Layers

 



 

TCP Client Server

 



 

TCP vs UDP

 

·         TCP is a connection-oriented protocol, whereas UDP is a connectionless
protocol.

·         The speed for TCP is slower while the speed of UDP is faster

·         TCP uses handshake protocol like SYN, SYN-ACK, ACK while UDP uses no
handshake protocols

·         TCP does error checking and also makes error recovery, on the other
hand, UDP performs error checking, but it discards erroneous packets.

·         TCP has acknowledgment segments, but UDP does not have any
acknowledgment segment.

·         When we compare TCP vs UDP protocol, TCP is heavy-weight, and UDP is
lightweight.


2.2   BASIC WEB ARCHITECTURE

 



 

HTTP features:

·         Request

o   Request URL – address of the resource requested on the Server

o   Request method: GET/POST/PUT, etc

o   Request Headers:

§  Client info (User-Agent, Referrer, etc)

§  Cookie & cache info

§  Accept info

o   Request Data (POST/PUT methods)

·         Response

o   Response code

§  1xx – information

§  2xx – success

§  3xx – redirect

§  4xx – client error

§  5xx – server error

o   Response Headers:

§  Server info

§  Content info (type, length)

§  Cookie & Cache

o   Response Data

 

 


2.3   HIGH LOAD WEB ARCHITECTURE

 




2.3.1  CACHING

 

 

First step on reducing the load on the Web Platform is using cache mechanisms.

 

Pros: increasing speed/response times, lighting the load on the platform, Cons:
users can be presented with obsolete data, higher memory/resources usage. To
reduce Cons as much as possible the cache should always be checked/configure to
limited memory and duration (automated expiration etc).

 

There are a lot of places/components where caching can be used:

·         Server side

o   Application cache – at the application level (ASP.NET, Java, etc) memory
cache (OOB or custom implemented):

§  Cache heavy accessed data from the database

§  Cache CPU consuming calculations

o   Output cache – at the Web Server level (IIS, Apache) – caching the data that
was sent by the server for specific URLs, so next time the same URL is
requested, the Web server will not send it internally to the backend

·         Proxy – traffic between Server and Clients

o   Usually these are dedicated caching server, are placed in front of the Web
platform, they have pretty similar functionality as the Output Caching

·         Client side

o   Browser caching – commonly used by all the browsers, when accessing a web
site, there are a lot of HTML/JS/CSS and media files. Depends also on Server
response headers (can be instructed not to cache, etc).

o   Application caching – similar to Server side, memory / timed caches can be
set from JavaScript

           


2.3.2  SCALING

 

Heavy traffic web platform must be able to always scale (ideally horizontally
and vertically):

-       Horizontal scaling – adding new machines to process in parallel the
traffic

-       Vertical scaling – adding more power to the existing machines (Web
Servers, Database): more memory, more CPU, more storage.


2.3.3  LOAD BALANCING

 

Again, on heavy traffic / high performance volume, you use Proxy servers for
load balancing. There are probably a dozen of load balancer platforms available
(Cloudflare, AWS, etc).

As the name says, a load balancer will take the incoming traffic and
analyze/decide to what server it will serve the request.


2.4   AUTHENTICATION & AUTHORIZATION

Authentication & authorization will apply to web platforms that provide specific
content and logic per user. For this, the platform will need to host and
implement logic for user accounts.

 

Authentication: validates that the users are whom they claim to be.
Authentication process: Passwords, PINs, Biometrics, Authentication apps

Authorization: defines the permissions allowed for an authenticated user.
Authorization process: access levels, roles, security permissions, etc.


2.4.1  WEB PLATFORM AS SERVICE PROVIDER AND IDENTITY PROVIDER

 



A basic Web platform will act as both Service Provider (SP), which is providing
the actual logic of the platform and IdP (Identity Provider), user
authentication service.

 

Major technology providers offer also out of the box membership implementation
(ASP.NET Membership, etc).


2.4.2  WEB PLATFORM AS SERVICE PROVIDER AND SEPARATE IDENTITY PROVIDER

 



Is a common practice for major platforms to allow users to authenticate with
their Facebook, Google, etc accounts. So this is an instance when the
authentication is carried out by a third party, the platform acts only as
Service Provider.

 

 

 

 


3     CHAPTER THREE – SERVER DETAILS

 

For our scope, we are defining a Server as a generic software application that
can processing incoming information from multiple Client applications over a
network. Of course, physical computers or cloud hosted computers are also
referred as Servers, but in our course, we are focusing on the software side.

 



 

Going into further detail we can identify 3 major components on a Server app:

 

-       Connection and communication

-       Processing

-       Persistence

 

 


3.1   CONNECTION AND COMMUNICATION

 


3.1.1  OSI MODEL

 

Most commonly the Server operates directly at the Transport (TCP/UDP) or
Application Layer (HTTP, FTP, POP, SMTP, etc). Keep in mind that the
communication at Application layer is also relaying on the Transport layer.

 


3.1.2  NETWORK SOCKETS

 

At a lower level, the connection and communication will rely on network sockets.
A network socket is a software structure within a network node of a computer
network that serves as an endpoint for sending and receiving data across the
network. The structure and properties of a socket are defined by an application
programming interface (API) for the networking architecture. Sockets are created
only during the lifetime of a process of an application running in the node.

 

Usually, a server operating at these layers will:

 

-       Listen for incoming connections at a specific port

-       Create communication connection for each client that will come to the
server

 

A server is identified through a network address - IP or hostname (which
internally is still just an IP) and a listening port.

 

 

 



 


3.1.3  DEFAULT PORTS

 

There are up to 65,535 ports that can be used. The ports from 0 to 1023 are
usually system ports and they are used by specific applications:

 

20, 21 – FTP

25 - SMTP

80 – HTTP

443 – HTTPS

 

Technically you can have server custom applications that can use those ports,
they are not forbidden, but that is not practical, and, if a Server already
listen to a port, there cannot be another one listening to that port.


3.1.4  WEB SERVERS

 

As in the definition above, a web server will typically listen on port 80 for
HTTP and 443 for HTTPS connections.

Although you can always build your own, in practice there are a couple of Web
servers that are used for most of the web implementations:

 

-       IIS (Internet Information Services) – web server provided by Microsoft.
Currently available only on Windows based Operating Systems (probably soon on
Linux / Mac as well).

As for the technology stack, the server is built to work together with the
Microsoft ASP.NET, which means the programming language supported is C# and
VB.NET and Microsoft provides its own ORM solution: Entity Framework that works
best with Microsoft SQL Server database.

For a real-life project, using IIS usually implies a few costs, given that most
of the Microsoft tools require licenses (IIS usually should run on a licensed
machine, Microsoft Visual Studio licenses). Together with the Microsoft Cloud
implementation (Azure), IIS is a powerful Web Server competitor.

-       Apache – open-source web server. Runs on all operating systems (Windows,
Mac, Linux, Unix).

Regarding the technology stack: highly configurable (but that comes also with
more complexity) it usually runs best with Java web technology and PHP. Supports
also multiple ORM implementations (Hibernate for Java, Doctrine/Propel for PHP),
multiple database servers: MySQL, PostgreSQL, Oracle.

For a real-life project, an Apache implementation using Java/PHP don’t occur any
licenses usage, further that that, if the Operating System is Linux, there are
no OS licenses needed as well (of course, we still have hosting costs and
machine usage costs).

-       NGINX – open-source web server and paid alternative that offers more
tools. Pretty similar to Apache, from the performance perspective NGINX is
better, however Apache supports way more configuration and tools.

 

 


3.2   PROCESSING

 

Here happens all the backend business logic of the Server application. The logic
starts after the HTTP request has been received from the Client and until a
response is prepared to be sent back to the Client (see the communication
component above).

 

In this component relies most the Backend implementation effort, based on the
chosen Web Server, Operating System and the Technology stack: ASP.NET & C#, Java
Spring, PHP, Node.js, etc.

 


3.2.1  USING APIS FOR CLIENT – SERVER INTEGRATION

 



 

The Client – Server implementation must rely on a common communication framework
on how to exchange the data between them. Application Programming Interfaces
(APIs) are constructs made available in programming languages to allow
developers to create complex functionality more easily. They abstract more
complex code away from you, providing some easier syntax to use in its place.

 

Of course, on Web platforms we will use Web APIs.

 

API protocols:

REST

SOAP

Works with XML, JSON, HTTP and plain text

Works with XML by design

Loose and flexible guidelines based on architectures

Strict, clearly defined rules

Modest security

Advanced security

Works well with data

Works well with processes (actions)

Uses low bandwidth and is highly scalable

Uses more bandwidth with limited scalability

 

 

Data format:

-       Json – commonly used on most modern applications, more compact, easier
to read

-       Xml – old format, also has defined structure

 

3.2.1.1   ODATA AND ANATOMY OF A HTTP REQUEST/RESPONSE

OData (Open Data Protocol) is defined as a standard that defines a set of best
practices for building and consuming RESTful APIs (https://www.odata.org/). Is
built on core protocols like HTTP and commonly accepted methodologies like REST
for the web.

 

It has a predefined standard of requesting and querying the data using specific
keywords, that are supplied as URL parameters in the request ($filter, $orderBy,
$select, etc).

 

The basics of the protocol are very good explained in their basic tutorial:
https://www.odata.org/getting-started/basic-tutorial

 

You can analyze an HTTP request using various tools, most common (and practical)
is using the Network tab of pretty much any of the major browser’s console.

 

Below we can see the anatomy of the HTTP request based on OData:

 

-       First tab showing: Request info (URL, params, HTTP method, headers),
Response info (Status code, headers):

 

 

 

DB Trainers -> API Trainers ->

GET Trainers ->

POST -> add train

PUT -> modific

PATCH -> modific

DELETE -> sterg trainer

 



 

-       Payload tab, showing decoded URL parameters, where we can see for our
example the OData query params

 



 

-       Response tab, showing the JSON data sent in the response from the
server:

 




3.2.2  INTERNAL API VS PUBLIC API

 

Usually, a web implementation will have two sets of APIs:

 

-       Main one, available for its own functionality and will support all the
functionality

-       Public API - External one, available to be used programmatically by
external clients/users. This set of API is limited as functionality, it doesn’t
expose all the platform functionality and is used usually to integrate the
current web implementation with other external implementations. You will find an
available Public API on all major web implementations. As a small example:
Facebook has its own Frontend and Backend implementation but also provides a
Public API that can be used for any developer to develop Facebook applications,
implement Login with Facebook on its own web implementation, etc.

 

swagger


3.3   PERSISTENCE

On a standard web implementation, the Server will need to have the possibility
to store user data, so that can be done using a database server.

 

The steps are:

 

-       First you need to identify what data you want to store for your
application. Based on this, implement the appropriate database tables. A small
example below for an application that would manage the scheduling for the
training classes of a gym:

 



-       Develop matching classes on the Backend implementation

-       Use a communication layer between the Backend implementation and the
Database server

-       Based on the implementation, add/edit/delete information from the
database

 

 


3.3.1  OBJECT RELATIONAL MAPPING (ORM)

 

Object Relational Mapping (ORM) is a technique (Design Pattern) of accessing a
relational database from an object-oriented language. All the modern web
application platform provides also ORM frameworks. Using an ORM framework, all
the steps above are done from the programming language used. Main features of an
ORM:

 

-       Very less interaction with the database

-       No need to use a communication layer with the database to run the
database instructions to process data (SELECT, INSERT, UPDATE, DELETE, etc)

-       Supports changes of the models and generates automatically update scrips
(Migrations)

 

Most common ORM’s: Entity Framework (C#) and Hibernate (Java).

 


3.3.2  RELATIONAL VS NOSQL DATABASE

 

Most of the steps above apply to Relation databases.

 

Relational databases have been a prevalent technology for decades. They're
mature, proven, and widely implemented. Competing database products, tooling,
and expertise abound. Relational databases provide a store of related data
tables. These tables have a fixed schema, use SQL (Structured Query Language) to
manage data, and support ACID guarantees.

 

No-SQL databases refer to high-performance, non-relational data stores. They
excel in their ease-of-use, scalability, resilience, and availability
characteristics. Instead of joining tables of normalized data, NoSQL stores
unstructured or semi-structured data, often in key-value pairs or JSON
documents. No-SQL databases typically don't provide ACID guarantees beyond the
scope of a single database partition. High volume services that require sub
second response time favor NoSQL datastores.

 


3.3.3  DATABASE PROGRAMMING

 

Usually using an ORM, a developer won’t have to use much database programming or
not use at all. Mostly, as a Developer and pretty much all other roles in the
project, you would:

-       Query the database for data

-       Manual correct data (using insert/update/delete SQL commands)

and you won’t change the structure, this would be handled through ORM.

 

However, there are cases when you will need a bit more support from the
database, that’s when doing a bit of programming will help. Most common used at
this level are:

 

-       Triggers – configured at table level, you can configure actions that can
happen on each Insert/Update/Delete. For example, you can have a trigger that
will update the change count, author on each table automatically.

-       Stored procedure – allow implementing programming logic, similar cu
functions/methods on other languages

-       Views - based on a query that runs on one or more database tables.
Database views are saved in the database as named queries and can be used to
save frequently used, complex queries

 

 

 


4     CHAPTER FOUR - CLIENT DETAILS

We are referring to any type of Client application connected to a Server over
various network protocols.

 

In general, we can split clients into 4 large categories:

 

1.    Desktop application

 

We will refer as client desktop application to any application that needs to be
installed on a computer, connects to a Server platform through various protocols
and is not accessible through a web browser. They can run on regular user
desktop machines, server machines, industrial devices, etc.

 

In a larger sense, even the browsers installed on a device are considered a
desktop application.

 

-       Email apps (Outlook, Thunderbird, etc), Database management tools (SQL
Management studio, etc)

-       Streaming apps usually using UDP; Spotify – proprietary network
protocol, but based mostly on TCP

 

2.    Device application

 

Like desktop applications, basically apps that can be installed on any type of a
smart device (TV, Fridge, etc).

 

(IoT)

 

Industrial apps – for example applications using lower protocols to control
movement of materials in a factory (reading information from scanners, sending
information to conveyors etc)

 

3.    Mobile application

 

Like desktop applications, of course using different technology stack for the
implementation.

 

 

4.    Web application

 

These are the applications accessed through a web browser.

 


4.1   WEB CLIENTS

 

We can also group the web clients into two main categories:

-       Web sites

-       Web applications/platforms

 


4.1.1  WEB SITES

 

Most common, pretty much the first web applications released:

-       Visitors mostly consume the content, perform few other
operations/interactions (usually just search, contact submit forms). Shopping

-       Usually for all type of presentations (business, blogs, brand
catalogues, etc), ideas, promotions, etc (pretty much most of the Internet).

 

Compared to old web sites, modern ones allow visitors to create user accounts
and interact more with the platform (shopping, submitting content, etc). Once
the user creates an account, the experience doesn’t change much, but even if is
not visible to the user they are usually redirected to separated web
applications/platforms (see next chapter), which have more user interaction
implementation.

 

Key considerations:

-       Focused on easy content entry by the platform owners, less on
interactions (using CMS).

-       Can use extensively caching mechanisms, the content changes at a smaller
pace, so most of the information can be cached for longer periods of time

-       SEO – optimized for search engines: HTML DOM structure, redirects,
keywords & content. Usually, you will want to be easily found by the search
engines

-       Multiple languages / locales. Multiple locales – locale domains (.de,
.fr, etc)

 

4.1.1.1   WEB SITE IMPLEMENTATIONS - CMS

 

Initially, the websites were developed mainly manually on HTML with very basic
CSS and JS logic. Next, various tools became available to visually build the
HTML files. Although it was an improvement, this would also imply that the
developer will had to maintain and upload the files to the web server, not to
mention that it would have been difficult to be maintained by a non-developer.

 

Modern websites are implemented usually using Content Management Systems (CMS).
A CMS will allow you to implement and usually deploy the content of the website.

 

There are a LOT of CMS platforms right now, below the are some key
considerations when choosing a CMS:

 

-       Technical knowledge of the developers/implementors:

o   WordPress, Wix: easier to use, but more complex on custom programming
extensions

o   Drupal, Sitecore, AEM: more complex to use but very customizable.

-       Existing technology stack used by the implementors. For example, if you
use Microsoft .NET, Sitecore, Umbraco CMS can be more suitable, etc

-       Multilanguage / multi-locale: web sites that have a lot of locales
(NIVEA – probably more than 80 locales, Michelin – same, etc) will need a CMS
that provides great support and optimizations on publishing content to multiple
languages.

-       eCommerce requirements – using CMS like Magento

 

4.1.1.1.1    WORDPRESS VS UMBRACO

 

Quick comparison between a user-friendly CMSs and a developer friendly CMS.

 

WordPress



Umbraco

 




4.1.2  WEB PLATFORMS / APPLICATIONS

 

With the progress of the web browsers and the JavaScript and CSS technologies, a
lot of old desktop applications are now running on Web.

Modern implementations allow implementation of various tools directly on web,
since the technology supports now using a lot of visual controls (buttons,
texboxes, grids, drag&drop, etc). One of the most known examples is the Google
Drive implementation of the Microsoft Office desktop applications (Google
Sheets, Google Docs, etc) which is full web.

 

On a web platform/application:

-       Visitors don’t just consume content; they add content and interact in
various ways with the existing content.

-       Visitors will have user accounts so that the platform can control the
access to the content creation/interaction

 

Key considerations, compared to a Web site:

-       Custom built using modern JS platforms since the interactions are
practically unlimited.

-       Content changes a lot since the users are continuously updating it, so
the caching data must be limited, so in order to achieve performance, the
implementation relies on fast API calls to get chunks of data.

-       Don’t care/use about SEO, to attract traffic you will have a separate
Web site for presentation of the platform which is SEO optimized.

-       Multiple languages / locales. Multiple locales – locale domains (.de,
.fr, etc)

-       Authorization & authentication

 

4.1.2.1   WEB PLATFORM IMPLEMENTATION

 

The typical implementation consists of a server-side implementation (Backend)
and a client-side implementation (Frontend). Some technologies / developing
platforms allow also to create both Backend and Frontend together (one single
implementation), like ASP.NET MVC.

 

Backend:

-       Here we will control the business logic of the application. Most of the
calculation, analysis is performed on the Backed

-       Will expose a set of APIs to the Web client. Not visible to the Users or
only specific APIs made public to specific users.

-       Will manage the data persistence with the database.

-       Commonly implemented on .NET, Java, PHP and an HTTP Server (IIS, Apache,
nginx).

 

Frontend

-       The application accessed by the users.

-       Highly interactive and uses the provided API for accessing &
manipulating data/content.

-       Implemented with JS based frameworks: Angular, React, ext.js for the
logic, CSS frameworks for styling (Tailwind, Bootstrap, Material UI).

-       Frameworks above allow developers to write high level code, object
oriented, strongly typed which will be compiled / packed to JS/HTML/CSS standard
code usable by any browser.

 

For a quick overview, see below a small overview of some of the most common web
frameworks, but first a small summary of the MVC design pattern which is
commonly used as well on both Backend and Frontend frameworks.

 

4.1.2.1.1    MVC PATTERN

Official definition: Model–view–controller (MVC) is a software design pattern
commonly used for developing user interfaces that divide the related program
logic into three interconnected elements. This is done to separate internal
representations of information from the ways information is presented to and
accepted from the user.

 

Traditionally used for desktop graphical user interfaces (GUIs), this pattern
became popular for designing web applications. Popular programming languages
have MVC frameworks that facilitate implementation of the pattern.
(https://en.wikipedia.org/wiki/Model–view–controller).



 

4.1.2.1.2    ASP.NET MVC VS ASP.NET WEB API

 

We are using ASP.NET as comparison, but the rules apply to pretty much all the
existing platforms right now, main choice is between a Server / Backend app that
will generate most of the platform or Server / Backend just providing data
(through APIs) and the platform UI build is on the Frontend. Historically, the
first option was initially most used, currently second option is most common.



 

Even they rely on the same libraries and tools, there is a significative
difference between these two main frameworks provided by Microsoft: ASP.NET MVC
will allow you to build complete a website (Backend and Frontend), ASP.NET Web
API will only allow you to build a set of APIs that will be consumed by a
separate Frontend application.

 

ASP.NET MVC

 

Base concept here is that the Backend framework will send fully generated HTML
or chunks of HTML view, directly to the browser, so all the HTML generation is
happening on the Backend, on the Frontend side the processing is done by the
standard Browser engine.

 

Pros: use just one technology and one implementation to build a complete
platform. Quicker development time.

Cons: more data traffic (the server will send always full HTML content which is
significantly larger than Json/Xml), more load on the Server (will become the
bottleneck since all the processing is done on the Server/Backend side), low
flexibility (will rely less on Javascript, since the views and logic will be
already sent to the Browser). More difficult to maintain/scale.

 

 

ASP.NET Web API and Frontend UI

 

In this case the Backend will provide just the data through APIs, usually in
Json/Xml format (Json - most common), and the UI is built on the Frontend. In
this case the application will be build on a strong mix of HTML & Javascript
with a lot of focus on Javascript (compared to case above where the focus is
more on HTML).

 

            Pros: Less traffic, less processing on the Server. Allows greater
flexibility on how the UI is built.

            Cons: Needs a separate Frontend platform for implementation.

 


4.1.3  VISITOR ACCESS TO A WEB SITE (DNS)

 

-      

 


5     CHAPTER FIVE – BEST PRACTICES AND STANDARDS

 

In a real-life implementation, the more practices and standards are implemented
AND followed by the team, the more it will ensure a successful outcome. Further
than that, usually one of the key differences between a Senior and a Junior
developer is the ability to use in the daily work these practices and standards.

 


5.1   LOGGING

 

As we can see, an application or an executable code can run in various ways and
platforms (backend API code that runs 24/7 on various servers, desktop
application, mobile application, frontend application that runs in a browser,
etc) and it is very important to have a way to trace the program execution. In
practice this is done through logging.

Through logging, we can detail the execution steps of the application, we can
trace values, errors, warnings etc and that information is usually written by
the application on simple text files. For example, since a backend API
application doesn’t have any visible interface, we have no way to know what
happens during an API call, or what happens if the API calls crash with error,
so using logging, we can just check in the log files all of that.

Logging requires a bit of extra development effort because the developers are
responsible for implementing code to log information.

 

Code sample:

 

 var fileName = "abc.txt";

            log.Info("Start reading file: " + fileName);

 

            try

            {

                using (StreamReader file = new StreamReader(fileName))

                {

                    int counter = 0;

                    string ln;

 

                    while ((ln = file.ReadLine()) != null)

                    {

                        log.Debug("Line " + counter + ": " + ln);

                    }

                    file.Close();

                    log.Info("File has " + counter + " lines.");

                }

            }

            catch(Exception ex)

            {

                log.Error("Could not process file: " + fileName, ex);

            }

 

Key points:

-       Configuration – most of the logging platforms are configurable in terms
of levels, appenders, formats. See below for more info.

-       Log levels – usually the information sent to the log has a priority
level, for example if is an error it will have a higher level than a simple log
information. The log levels are configurable, so for example, on production
environments the applications will be configured to log only errors, warnings.
Common log levels: DEBUG/VERBOSE, INFORMATION, WARNING, ERROR, CRITICAL.

-       Log appenders – an appender is a configurable log target. For example
you can have log information going to a file (through a file appender) or you
can build a network appender and send that information to a log server.

In the test configuration below we have two appenders:

o   One to write all logging information on text files, with max size 10 MB and
when the file size exceeds 10 MB, create a backup file, up to 100 backups

o   One to write only errors and warning to a custom appender

 

<log4net debug="true">

    <appender name="RollingLogFileAppender"
type="log4net.Appender.RollingFileAppender">

      <file value="App_Data\\app.log" />

      <appendToFile value="true" />

      <rollingStyle value="Size" />

      <maxSizeRollBackups value="100" />

      <maximumFileSize value="10MB" />

      <staticLogFileName value="true" />

      <layout type="log4net.Layout.PatternLayout">

        <conversionPattern value="%-5p %d [%thread] %5rms %-22.22c{1} %-18.18M -
%m%n" />

      </layout>

    </appender>

    <appender name="WorkflowAppenderError"
type="WorkflowLogAppender.WorkflowAppender, WorkflowLogAppender">

      <layout type="log4net.Layout.PatternLayout">

         <conversionPattern value="%-5p %d %5rms %-22.22c{1} %-18.18M - %m%n" />

      </layout>

      <filter type="log4net.Filter.LevelRangeFilter">

         <levelMin value="ERROR" />

         <levelMax value="FATAL" />

     </filter>

  </appender>

</log4net>

 

 

-       Formatting – you can also configure the log file structure, what fields
to be displayed, for example in the log file below we want for each line of
logging to display: log level, timestamp, thread id, class name, method name,
log information.

 

INFO     2023-04-20 14:50:25,020 [97] FileControlller      
ReadFile               - Start reading file: abc.txt

DEBUG 2023-04-20 14:50:25,034 [97] FileControlller       ReadFile              
- Line 0: test

…

 

Most common log platform – log4net.

 


5.2   ERROR HANDLING

The developer must take into consideration how to handle any application issue:

-       Error or exception: most modern programming platforms support exception
handling (try/catch).

-       How the application will behave:

o   Recoverable errors/exceptions

§  They are logged

§  Usually the application will inform the users that there was an issue

o   Make sure all exception are caught, usually through a general handler

o   Critical errors that can bring the system down – use an external
notification (email, sms, etc)

 


5.2.1  WEB ERROR HANDLING

 

A web application should consider having at least these two handlings in place:

-       A not found page (404): when the user request an URL that is not found,
the application should display a custom page telling the user that the content
is not found

-       An error page: when the user requests a valid URL, but the
server/application somehow crashes executing the request, the user should be
redirected to a friendly error page

 


5.3   CODE STYLE AND FORMATTING

Since there are always multiple people working on a project, we must ensure that
we have code clarity and each person code should match the general style and
formatting rules.

 

Coding style conventions:

-       How to name variable / fields / properties: lower/upper camel case (eg:
startDate, endDate), using an underscore in front of the variable (eg:
_startDate, _endDate)

-       How to name functions, properties, classes

-       There are some styles already standard worldwide, such as Hungarian
notation, but in the end is more important that there is a standard and used by
all the team members. So having a few people using one standard, others using
another one, SHOULD BE AVOIDED.

 

Code formatting:

-       Same number of spaces for Tabs

-       How to define block of codes – brackets on the same line with the
condition or new line:

o   If (condition) {

….

}

o   If (condition)

{

…

}

Most of the IDE’s provide automated formatters, for example you can set rules
that at each Build the code to be automatically formatted.

 

Even though it seems simple, in practice, not using this led to a lot of
misunderstanding on the code between the team members.

 


5.4   CODE ANALYSIS

 

Usually when building the code, any compiler will display a set of errors and
warnings. Additional to that, most of the IDE’s provide separate code static
analysis tool that also check your code quality. Usually they are checking for
security, performance, design but also other issues.

 

This code analysis tools are also highly configurable. So, using the full code
analysis in practice, sometimes can slow down / complicate the implementation.
Is recommended that the code analysis should be configured to specific set of
rules.

 

 


5.5   CODE REVIEWS

 

A must on highly critical industries (aviation, automotive), but also a good
practice in general, especially for Junior / Mid level developers: before a
piece of code is submitted to the code server, the developer will show it,
explain it to other developers from the team.

Of course, the code repository platform offers various ways for that, showing
the code differences, pull requests.

 

 

SCREENSHOT PULL REQUEST

 

 


6     WEB DEVELOPMENT COMMON FEATURES

 


6.1   COOKIES

The main definition from Wikipedia is straight forward:

“HTTP cookies (also called web cookies, Internet cookies, browser cookies, or
simply cookies) are small blocks of data created by a web server while
a user is browsing a website and placed on the user's computer or other device
by the user's web browser. Cookies are placed on the device used to access a
website, and more than one cookie may be placed on a user's device during a
session.

Cookies serve useful and sometimes essential functions on the web. They enable
web servers to store stateful information (such as items added in the shopping
cart in an online store) on the user's device or to track the user's browsing
activity (including clicking particular buttons, logging in, or recording
which pages were visited in the past)”

Usage example:

 




6.2   BROWSER STORAGE

 

A web application can also store some data in the local storage using JavaScript
API. There are several reasons why many of the websites and apps we come across
store some data locally in the browser. The major reason associated with browser
storage is performance. The data stored locally in the user's browser is
instantaneously available, and on the other hand, the remotely stored data is
sent from the server to the user.

 

Some common examples:

 

-       Storing application preferences

-       Saving/caching server data that is static or is changed very rarely

-       Saving last accessed url, username etc

 

Web browsers supports two different storages:

 

-       Local storage object - Stores data with no expiration date

-       Session storage object - Stores data for one session (data is lost when
the browser tab is closed)-


6.3   CACHING

 

 


6.4   AUTHENTICATION AND AUTHORIZATION

Example flow, cookie usage.

 

 

 



 

 

 

 

 


7     TESTING

 

Development and testing are used together through the entire lifecycle of the
project.

 


7.1   TYPES OF TESTING

 

1.    Unit tests

Unit tests are very low level and close to the source of an application. They
consist in testing individual methods and functions of the classes, components,
or modules used by your software. Unit tests are generally quite cheap to
automate and can run very quickly by a continuous integration server.

 

2.    Integration tests

Integration tests verify that different modules or services used by your
application work well together. For example, it can be testing the interaction
with the database or making sure that microservices work together as expected.
These types of tests are more expensive to run as they require multiple parts of
the application to be up and running.

 

3.    Functional tests

Functional tests focus on the business requirements of an application. They only
verify the output of an action and do not check the intermediate states of the
system when performing that action.

 

There is sometimes a confusion between integration tests and functional tests as
they both require multiple components to interact with each other. The
difference is that an integration test may simply verify that you can query the
database while a functional test would expect to get a specific value from the
database as defined by the product requirements.

 

4.    End-to-end tests

End-to-end testing replicates a user behavior with the software in a complete
application environment. It verifies that various user flows work as expected
and can be as simple as loading a web page or logging in or much more complex
scenarios verifying email notifications, online payments, etc...

 

End-to-end tests are very useful, but they're expensive to perform and can be
hard to maintain when they're automated. It is recommended to have a few key
end-to-end tests and rely more on lower level types of testing (unit and
integration tests) to be able to quickly identify breaking changes.

 

5.    Acceptance testing

Acceptance tests are formal tests that verify if a system satisfies business
requirements. They require the entire application to be running while testing
and focus on replicating user behaviors. But they can also go further and
measure the performance of the system and reject changes if certain goals are
not met.

 

6.    Performance testing

Performance tests evaluate how a system performs under a particular workload.
These tests help to measure the reliability, speed, scalability, and
responsiveness of an application. For instance, a performance test can observe
response times when executing a high number of requests, or determine how a
system behaves with a significant amount of data. It can determine if an
application meets performance requirements, locate bottlenecks, measure
stability during peak traffic, and more.

 

7.    Smoke testing

Smoke tests are basic tests that check the basic functionality of an
application. They are meant to be quick to execute, and their goal is to give
you the assurance that the major features of your system are working as
expected.

 

Smoke tests can be useful right after a new build is made to decide whether or
not you can run more expensive tests, or right after a deployment to make sure
that they application is running properly in the newly deployed environment.

 

 


7.2   MANUAL VS AUTOMATED TESTING (WEB)

 

All the types of testing are done manually, but for some of them (functional,
end-to-end, smoke testing mostly) we can use automated testing platforms.

Specifically for web, an automation testing platform will allow the user to
instruct it to test and validate results in a browser pretty close to how a real
person will test: interact with web controls (click, edit text, etc), check the
page behaviour (eg when clicking a button, make sure a specific section appeared
or browser has redirected to another expected page, etc).