Saturday, January 21, 2012

A journey to Delphi TParams - Persistence

In the first article I demonstrated some features of TParams we can use at every day development to hold, process and move data structures around. But what if we want to persist such data and come back to them later?

As you will see, persisting is very easy and is based on Delphi’s component streaming mechanism. It is the same core mechanism used by IDE to save and load a form’s design in .dfm files. TStream class introduces methods that work in conjunction with components and filers for loading and saving components in simple and inherited forms. The two TStream methods needed are ReadComponent & WriteComponent and their definitions are:

function ReadComponent(Instance: TComponent): TComponent;
procedure WriteComponent(Instance: TComponent);
Delphi help also introduces two example functions to show how to use the built-in component streaming support to convert any component into a string and convert that string back into a component. These functions are a long time ago in my utilities library!

function ComponentToString(Component: TComponent): string;
var
  BinStream:TMemoryStream;
  StrStream: TStringStream;
  s: string;
begin
  BinStream := TMemoryStream.Create;
  try
    StrStream := TStringStream.Create(s);
    try
      BinStream.WriteComponent(Component);
      BinStream.Seek(0, soFromBeginning);
      ObjectBinaryToText(BinStream, StrStream);
      StrStream.Seek(0, soFromBeginning);
      Result:= StrStream.DataString;
    finally
      StrStream.Free;
    end;
  finally
    BinStream.Free
  end;
end;

function StringToComponent(Value: string): TComponent;
var
  StrStream:TStringStream;
  BinStream: TMemoryStream;
begin
  StrStream := TStringStream.Create(Value);
  try
    BinStream := TMemoryStream.Create;
    try
      ObjectTextToBinary(StrStream, BinStream);
      BinStream.Seek(0, soFromBeginning);
      Result := BinStream.ReadComponent(nil);
    finally
      BinStream.Free;
    end;
  finally
    StrStream.Free;
  end;
end;
Now that we have all the streaming functionality in our hands we can stream in & out a TParams collection, or we cannot?

No, we cannot. Component streaming works with TComponent & descendants and TParams is not one of these, it actually derives from TPersistent->TCollection.

But this is something easily fixed just by declaring a TComponent with a published TParams property. The streaming mechanism can then deal with this component and “magically” save and load the TParams collection.

Here is such a declaration, very simple and clean:

TParamsStorage = class(TComponent)
protected
 FParams: TParams;
published
 property Params: TParams read FParams write FParams;
end;
Now that we have the ability to write and read a TParams collection to and from a string, we can persist it anywhere we want, a local variable, a file, a stream, a database field etc. I personally have a function and procedure to automate the process of converting TParams to string and vice versa. Here they are:

function ParamsToString(Params: TParams): string;
var ps: TParamsStorage;
begin
  ps := TParamsStorage.Create(nil);
  try ps.Params := Params;
   Result := ComponentToString(ps);
  finally ps.Free;
  end;
end;
procedure StringToParams(Value: string; Params: TParams);
var ps: TParamsStorage;
begin
  ps := TParamsStorage.Create(nil);
  try ps.Params := Params;
   ps.Params.Clear;
   StringToComponent(Value,ps);
  finally ps.Free;
  end;
end;
Another interesting effect of having a TParams collection in a string is that we can store this string in a single TParam object, effectively creating a tree structure of TParam collections!You can investigate the whole idea in the code behind the recursive function I use to create/update such structures:

TParamProps = record
  Name: string;
  DataType: TFieldType;
  Value: variant;
end;
function CreateTParams(const aParams: array of TParamProps): TParams;
var i: integer;
begin
  Result := TParams.Create;
  for i:=0 to High(aParams) do
   Result.CreateParam(aParams[i].DataType,aParams[i].Name,ptUnknown).Value := aParams[i].Value;
end;

function UpdateParam(
Params: TParams; const //The root TParams collection
Path: array of string; //TParam names hierarchy path
             FldType: TFieldType; //DataType of TParam to create
Value: Variant //Value of TParam to create/update
): Boolean; //True always 
var WP: TParams;
    P: TParam;
    A: array of string;
    i: integer;
begin
  if Length(Path) = 1 then
     begin
     P := Params.FindParam(Path[0]);
     if not Assigned(P) then
        P := Params.CreateParam(FldType,Path[0],ptUnKnown);
     P.Value := Value;
     Result := True;
     end
  else
     begin
     P := Params.FindParam(Path[0]);
     if not Assigned(P) then
        P := Params.CreateParam(ftString,Path[0],ptUnKnown);
     WP := TParams.Create;
     try if P.AsString <> '' then
           StringToParams(P.AsString, WP);
      SetLength(A,Length(Path)-1);
      for i:=0 to Length(A)-1 do A[i] := Path[i+1];
      Result := UpdateParam(WP,A,FldType,Value);
      P.AsString := UtilDB.ParamsToString(WP);
      finally WP.Free;
      end;
     end;
end;
Have fun developing, because development is fun!
Feel free to modify the above code as per your needs and if you make any enhancements please contact me.

Monday, January 2, 2012

Data Centric Development Framework

An application framework consists of software used by developers to implement the standard structure and base functionality of an application for a specific development environment and/or application needs. Developers usually use object-oriented programming techniques to implement frameworks such that the unique parts of an application can simply inherit from pre-existing classes in the framework.

Through my experience on facing requirements while building data centric applications for clients, i found out that there are common used constructs and need for utilities. So started gathering the tools and libraries i had built around, created new modules, classes and components and modified them so they could work together. Finally something came out of this; a development framework that could support almost all the base functionality i needed and the consistency i wanted my products to have.

My framework is a software development framework, fully object oriented and data centric, for the Delphi development environment that targets the Win32/64 platform and SQL RDBMS. It is designed for rapid application development and focuses on efficiency, usability and consistent functionality of the final product.

User experience, with intensive all day work on data entry and query forms, is among the targets of this development framework, which provides a well balanced UI that helps user productivity. The framework provides forms and data modules with common functionality for almost every kind of data, utility dialogs for every day user work and common behavior of UI and controls.

Combines together many tools, functions, components and user enhancements in an environment that most of its functionality is almost out-of-the-box, helping the developer to rapidly create the base of consistent data centric applications and then focus to the specific requirements of the user i.e. business rules and further UI enhancements.

The main technical features consist of an MDI project template, base form/report classes and data module classes, all with integrated functionality that can be customized via inheritance, component events, virtual methods and class functions. It also contains application and user configuration modules and a lot of tools, dialogs & components that act at runtime in conjunction with all main classes of the framework.

I spent many hours (days, months…) working on this in parallel with other projects and I still make enhancement and adding new features, but finally seems like it worth the investment!
Know i am in the process of upgrading & enhancing it for the new Embarcadero Delphi XE2 and hope i will come back soon with a full technical description.

So stay tuned and have very good year 2012 full of happiness and joy.


Following is a small non-exhaustive descriptive list of the framework’s main features:

User Authentication

User authentication can be set by groups of users and authorities can be granted or denied for form or application defined procedure level.

User Authentication User Authentication User Authentication


Data features

Supports ADO enabled databases and especially SQL Server. Can install and update database via external scripts. Handles internally connection string and other connection parameters. Automatically marks with create/change user & timestamp info the database records that support this feature. Supports transactions, automatic or manual, on record or batch basis. Provides logical data schemas to forms and reports and handles communication of data between different active forms.


Application Desktop - MDI Environment

The application desktop is an MDI controller form class with user customizable main menu and sidebar.
Each user form works inside and controlled by the main application form. User can activate, deactivate, hide, minimize, arrange and switch between client forms inside the application desktop.

Application Desktop - MDI Environment Application Desktop - MDI Environment Application Desktop - MDI Environment

Common User Access

The framework adopts this IBM’s standard (CUA) which was a detailed specification and set strict rules about how applications should look and function. A major set of rules refers to the standardization of keyboard shortcuts (F keys and combinations) that user can use to access common features. Each keyboard shortcut has an application wide meaning so users are never confused when working with the different application forms. Such common keystrokes are F1 for help, F2 for field editing, F3 for search, F4 for lookup, F5 for refresh and so on. The CUA standard was also the basis for the Windows Consistent User Interface standard (CUI), as well as that for OS/2 applications. Most of the standard keystrokes and basic GUI widgets specified by the CUA remain a feature of Windows, but few users are aware of them.


Lookup & Select

Wherever a reference to a field is required user can activate a helper window that let him search and select a value for this field, i.e. when a customer must be selected for a purchase order the user is presented with a list of available customers in a fully customized data grid. Features of filtering grid data and quick locate column value are available also here as almost in all other data grids.

Lookup & Select Lookup & Select Lookup & Select

Data Grids

Data grids are extensively used in many places of the application framework. Data entry forms, query results, lookup&select are some of them. Data grids can be customized in many ways such as visibility of columns, column grouping and summaries. They support common actions like filtering, locate column value, print as is and can be exported to html, csv, xml or text.


Data Filtering

Filtering is a must have for every serious data centric application. When dealing with large amounts data filtering is an essential way to isolate and easily locate those of interest. The framework has extensive support for filtering with multiple different filters per dataset which can be application and/or user defined.

Data Filtering Data Filtering Data Filtering

Dynamic data panels

Dynamic data panels can present data from a dataset record via dynamically created and/or runtime designed controls. Design specifications of the data panels can be saved for runtime reusability. As long as data logic is separated from the data presentation, a single form class which features dynamic control creation is enough and gives consistent UI experience.

Dynamic data panels Dynamic data panels Dynamic data panels

Context Sensitive Popup Menu

Popup menu is a feature widely used in Windows and the framework utilizes this feature with context sensitive popup menu on its forms and user dialogs. It helps user to select from available actions on a form by just right clicking on it.


Form Menu

Each form has its own menu with all the available functions that are defined by the application. The menu supports floating and dockable bars and is fully customizable.


Edit record template

Base form class that supports editing of a single database record in a data panel which can be dynamic. Can also be used as generic record editing form.


Browse/Edit file template

Browse base form class that supports browsing database tables, queries and dataset results in a data grid. Features filtering and all data grid functionality except editing. Can also be used as generic dataset browsing form.
Edit base form class much like Browse file template that supports editing multiple database records in a data grid. Features data filtering, and all data grid standard functionality. Can also be used as generic dataset editing form.

Browse/Edit file template Browse/Edit file template Browse/Edit file template

Reporting template

Base report class that support consistent reporting features and functions along the application. User can preview reports, export report to various formats, change printer settings and define font sets for specific reports.


Query executor

Utility to build and execute queries against the data base. Results are presented in a data grid with capabilities like browse file and also can be used by Report Generator. Query definitions can be saved for later use and can also be parameterized.

Query executor Query executor Query executor

Report manager

Reporting utility that features runtime report designer, save and load reports, custom layouts and reports based on user defined queries or datasets derived by the application like the ones manipulated by a Browse/Edit file templates or produced by Query executor.

Report manager Report manager Report manager

Chart generator

A chart generating form class that can get input from any application dataset and produce on the fly charts (pies, bars, lines etc) on screen and printer.

Chart generator Chart generator 
Chart generator Chart generator
Chart generator Chart generator Chart generator