Friday, May 30, 2008
Evils of debugging tiered software
I just spend about 3 hrs trying to solve a bug that i introduced while refactoring. I am refactoring some code in a pair of collaborating objects a listener and a service .
I am moving bus. logic from the listener into the service. My listener instantiates the service in its configure method. I created the problem in the one of the service functions, but the error was being reported by CF in the listener constructor at the line number where the instantiation to the service was being called. Eventually, i was able to track the error down in the service function that was using the same variable name for both an argument and a locally vared variable. The error was reported correctly but it took me way too long to figure out that the reporting of the error in the listener constructor was caused by a function in the service. Seems obvious in hindsight, that the listener constructor was failing because it was creating the new object that had to parse the entire cfc before it would create a new instance of itself.
Moral - if the error is occurring in a constructor or init function, then look carefully down stream into the instantiated objects.
Tuesday, May 27, 2008
Listener -vs- service layer
Here are a few excerpts from some respected machii architects, Sean Corfield and others.
"If you built a service layer that is independent of the framework that orchestrates actions taken against the model and then you built one or more listeners that mostly just delegate to that service layer, you'll be in good shape."
"Don't think it won't happen to you! I guarantee that you'll find yourself putting a Flex front-end on a legacy Mach II application at some point and you'll wish you too had done a better job of separating your business model from your listeners!"
"Right, a web service front end will also show the same issues. And just to be clear, Flex, Flash and Web Services all rely on the model portion, the CFCs, and not Mach II (or Model-Glue) which is why separating the framework from the model is so important."
See full discussion here.
This is from the machii developers guide on listeners:
Mach II interacts with the Model portion of your application using components that extend MachII.framework.Listener. In a simple application, all of your business logic might be implemented in such listeners but this approach does not scale well with increasing application complexity because, amongst other things, it introduces a tight coupling between the elements of your business logic and the Mach II framework.
You are much better off ensuring there is only a thin layer of coupling between the framework and your application - if necessary, create new components that extend the Mach II listener component, whose sole purpose is to communicate those events to the components within your business model. In other words, try to isolate your business components from the framework components as much as possible: only listener components should know about the framework; only listener components should access Mach II properties; only listener components should announce events. The core business components should know nothing about Mach II and have no dependencies at all on the framework.
In our cfobjective class last month we had a spirited discussion about this. One of the principles i recall was the listeners job being to pass along data to the service layer. The only logic that should be in the listener is logic pertaining to program flow. The listener could announce other events, reference controller properties and set event arguments. When it was done doing these things, it simply handed the event args off to the service. The listener knows nothing about the model layer and the model layer knows nothing about the listener, excepts the names of methods exposed via the API.
Design Patters - more details
Content paraphrased from http://www.jroller.com/page/kwiersma.
I keep coming back to these patterns because I support a large application that implements all of them and have been asked to share my knowledge. This is the knowledge! Granted, there is a lot of business logic in the application, but the architecture of the application follows closely the the principles found in these patterns.
Beans ( Business Objects or Value Objects)
DAOs (Data Access Objects)
Gateways (Data Gateway Objects)
Services (Manager Objects)
The Bean
- Typically represents a specific entity in your model
- Carries "encapsulated" data between the different layers of your application
- like a structure, instead of directly accessing data with a key you call a method
- Helps organize data structures instead passing structures around ad-hoc
- Has a consistent and simple interface (Controllable API)
- Has methods called getters/setters (aka accessors) to access data [getFirstName() / setFirstName()]
- Might be composed of other beans
- Easily created with a code generator
DAO (data access object)
- DAOs only interact with one row of data via the primary key
- Used to save/load objects from data storage
- A DAO could interface with: Database / Legacy persistent data storage (XML / Text File)
- Usually have ScRuD method that take a bean
- (ScRuD - Save | (create) | Read | (update) | Delete)
Gateways (Aggregated Data)
- return one or more rows of data
- Typically returns a cfquery object
- Rarely it can return a bean (Querying by User/Pwd)
- Performs functionality that may affect one or more rows
- Contains your business logic:
- Bean Validation / Creation
- Application specific business logic
- Save / Load / Delete from DAO (via the facade pattern)
- Gateway Interaction (via the facade pattern)
- Usually depends on a DAO and Gateway
- Services can consume other services (very social)
Monday, May 12, 2008
Value objects, Services and Objects Frameworks
Suppose in your application you deal with assessments, standards and resources. Each of these entities would be modeled by a value bean and each of the value beans would have their own service which would have its own DAO and Gateway objects.
A quick review of the value beans reminds use that each has its own constructor method that returns a reference to itself (passing by reference). This provides a complete blue print of the object, with attributes and methods to the requesting entity. Best practices suggest requiring as little as possible to instantiate a object. Define the arguments as optional with default values. Each value object has matching getter and setter methods for each attribute defined by the value bean. The setter methods do not return any values or references, they simply update a instance value. The getter methods return a granular value, string, integer, boolean (passing by value).
A quick review of a service object reminds us that the constructor method also returns a reference to itself. The constructor method would likely have object instantiations to the dependent DAO and Gateway objects and methods to call the DAO and Gateway methods. The service would likely have a utility methods that would create a new instance of its value bean
, which would return a object reference. The service may also contain other utility type methods like data validation on a form. The service methods will likely contain your business logic or rules and may need to reference data in other value objects. Remember to think of your services as the social entities in your software ecosystem. They are the glue between your application and model layer and contain most of the business specific rules that your objects must obey.
In this example, suppose the assessment service needs to know something about the standard bean, it would need to talk with the standard service, now the assessment service needs to instantiate a pointer to the standard service. Suppose the assessment service also needs to know something about the resource bean, now the assessment service must instantiate a pointer to the resource service. This type of dependency is common and can become difficult to manage and debug when things go wrong. Architects often ask who's job is it to keep track of these things? Why does service a need to create pointers to service b or c and vice versa for objects b and c needing pointers to services a and b. This is complex and tricky and has given rise to model layer managers. The model layer managers, like ColdSpring or Livewire abstract knowledge from the service layer. The model layer manages the creation of objects and their dependencies. These objects are all singletons and need only be created once during a given session.
Friday, May 9, 2008
design patterns - A walk through
This week, I am walking through a sample application provided by the good folks at cfobjective. I have my eye on the familiar data patterns and how they are implemented in a typical CF application. The application is not using a model manager like ColdSpring or LiveWire, nor is it using a application framework like machII, fusebox or coldbox. You could say its a little more clean this away and allows you to focus on the design of the objects (beans) and its interactions with its service, the application and model layer.
The application is following the MVC pattern and is using a service layer to interact with the DAO and Gateway and a facade to encapsulate the session scope.
The application.cfm file is creating the service object and other app. level setup work. The creation of the service object in the application scope allows it to be avail and used for the creation of a value or bean object and subsequent writes to the persistence layer via the DAO and Gateway.
The init() method of the service object is returning a reference to itself after it creates objects for both its DAO and Gateway. This is a case of the service object having state and knowing about other objects. If there was a model management tool, it would be responsible for keeping the state or dependencies, also referred to as CFC wiring.
The service contains all the methods to both the DAO and the Gateway. The service also contains the method to create a new value bean, which returns an instance of itself. In other words, the service is not only responsible for speaking to the data layer, but also for creating instances of the value beans that it is servicing.
If you need to create a value bean then find its service and call its public method that returns an instantiation of the bean. You are not returning just data like a string or a query, but the entire object, including the methods that are avail. in it. If you need to reference or update part of that value bean, then use its avail. getter or setter methods. This is clean object notation and usage.
For example, lets say we are dealing with a person bean or object.
A user has supplied details via a webform, those details have passed the submission filters and a new person object has been created and its attributes updated via bean setters.
cfset variables.person = CreateObject("component", "Person").init()
cfset variables.person.setFirstName(form.firstName)
cfset variables.person.setLastName(form.lastName)
cfset variables.person.setAge(form.age)
Next, call a addNewPerson function in the service, passing in the object reference and do a little error checking prior to calling the DAO function;
cfset variables.errors = application.personService.addNewPerson(variables.person)
If no errors result, call the addNewPerson function and require the object be supplied.
cffunction name="addNewPerson"...
cfargument name="person" type="Person" required="true" ...
If everything is in order, call the createPerson function, passing in the object reference, which will call the related DAO function.
cffunction name="createPerson" access="public"...
cfargument name="person" type="Person" required="true"
cfreturn variables.dao.create(arguments.Person)
In the create function of the DAO, require the person object be passed in and insert the record using the getter method for the person to get the actual value to write into the table.
cfargument name="Person" type="Person" required="true"
cfqueryparam value="#arguments.Person.getFirstName()#
cfqueryparam value="#arguments.Person.getLastName()#
This is a change of thinking for me, not easy, since i have treated CF in a procedural manner, where i am simply passing in args by value, not by object reference. I am trying to get my head around the benefit, which is trying hard to hide itself behind the added complexity, maybe its not really more complex, im just conditioned to think the other way and so think it more complex.
Thursday, May 8, 2008
Design pattern DAO -vs- Gateway
nuggets as follow
right off the bat, we realize this is not trivial stuff,
"Designing object-oriented software is hard, and designing reusable object-oriented software is even harder."
define a pattern:
"An object that encapsulates access to an external system or resource."
"Pattern: Bridge. Intent: Decouple an abstraction from its implementation so that the two can vary independently."
I agree with Matt Woodward, who says;
"The DAO and the Gateway, from a pattern standpoint, serve exactly the same purpose. They both encapsulate access to a database. The Gateway pattern is used by many ColdFusion developers to contain methods that handle multiple records. The DAO, on the other hand, is used in many ColdFusion applications to deal solely with single records; many ColdFusion DAOs contain only create(), read(), update(), and delete() methods (affectionately referred to as "CRUD")."
This is an excerpt from the mach-II devlopers guide, this version written by Sean CorField and provides great differenciation of the gateway and DAO design patterns and how they should be used. I have struggles to see this, even to explain it, but Sean seems to nail it down very well. Specifically the part about returning query sets from gateways but objects from DAO.
--- Begin quote from the Mach-II Development Guide ---
There are two basic patterns of access to persistent data within most applications:
- aggregated access - reporting, searching, listing multiple rows,
- per-object access - creating, editing, working in depth with a single row (object).
ColdFusion has a great built-in idiom for dealing with the first type of access - the query - which is an efficient way to manipulate (potentially large) sets of data rows retrieved from a database (or other data sources, since you can easily create a query object and populate it with your own data). When you are dealing with aggregated access, it does not make sense to convert every row returned into a fully-encapsulated object (CFC instance) when all you are likely to do with the data is display a few fields with a link to a detail page that will focus on the selected row.
On the other hand, when you are focusing on a single row it usually does make sense to work at the fully-encapsulated object level since you are usually interested in object behavior at that point. This is also the level where you need the standard CRUD (Create, Read, Update, Delete) operations.
Recognizing these two basic patterns, you should design your components accordingly by providing separate components for each pattern. This is best explained through an example:
- If we have a business model object called Order, then we would provide an OrderGateway component for aggregate access and an OrderDAO component for per-object access (or build the per-object access into the Order object - but see below).
- The OrderGateway component would provide methods like findAll(), findWhere(), findByID() and they would all return standard query objects (even findByID() which returns a single row).
- The OrderDAO component would provide CRUD methods like store(), load(), update(), delete() and they would operate on a specific Order object exchanging data via getters/setters on the Order component or via some sort of snapshot of the Order's data, e.g., a bean: the Order component could implement methods like getSnapshot() returning a bean and setSnapshot() taking a bean as an argument - the bean containing the core persistent data for the Order object. Sometimes a more direct data transfer between the business object and the data access object is needed for performance reasons (or because the application can 'trust' the components to exchange less encapsulated data, such as a struct or some opaque data structure e.g., the Memento design pattern). Such optimizations are beyond the scope of this document.
Separating out these operations from the business model object helps it stay persistence-neutral. The gateway components can be optimized for retrieving large record sets, caching etc. The DAO components can be optimized for dirty data updates, pooled object access and so on. Again, the details of these optimizations are beyond the scope of this document but providing for the two distinct patterns of data access will get you started on the right road.
--- End quote from the Mach-II Development Guide ---This wisdom i am paraphrasing from Matt Woodward blog entry, read entire post.
I like knowing that my DAOs deal with CRUD and that's it.
Gateways, on the other hand, are far more nebulous and end up changing much more during application development. I find that I add, remove, and alter methods in my Gateways pretty regularly right up until the application is complete. My DAOs, on the other hand, are built at the outset and they just sit there doing their job.
I also like knowing that when I call methods in my DAOs, they take in an instance of the object with which they are concerned (i.e. the bean) and don't return anything since they're operating directly on the objects passed in. Gateways, by contrast, typically return query objects or, when necessary, collections of beans. Some suggest that if you don't return a collection of objects, we aren't using this pattern "properly." Says who? Fowler? Sun? Maybe in Java the convention is to return a collection of objects. In ColdFusion we have the query that is efficient and easy to use. If all I want to do is loop over a query and display the data, and I don't need to interact with each individual object, why on earth would I make things more complex and less efficient just so I'm better meeting another technology's definition of how a Gateway is supposed to behave? Pure nonsense. When I need to deal with each individual object in an aggregate dataset, I can quite easily return an array of CFCs. When I don't need to do that, I won't bother. Some call that bad programming; I call it practical.
A paraphrase of a Sean Corfield post as response to this post,
"I still maintain that in ColdFusion it makes sense to keep per-object operations separated from aggregate operations because ColdFusion provides a wonderful abstraction called "query" :)
With the rise of ORM frameworks, it makes even more sense to keep "DAO" and "Gateway" separate because the ORMs replace the DAOs completely but only partially replace Gateways, if at all."
Leveraging Design Patterns
Do not be discouraged if you do not immediately see the payoff or even the need for a pattern at first, these are not always obvious and take time to internalize before you see them or the payoff in following or implementing one.
ColdFusion has evolved into a OO language that lends itself to design patterns. If you use cfinclude
Singleton Pattern - Some objects only need to be created once during the life of the user session, objects like services or application setup services are called singletons. They do not have to keep any state. For example, if i am tracking site visits in a log file or DB, i would only want a single record of that visit, not each time the user does something during their visit. When a user logs into a system, their information is retrieved and stored only once during their visit.
Factory pattern - Just give me what i need and its dependencies. I build objects and their dependency objects. If i am creating a DAO or Gateway object, then i will create also its service object.
Facade pattern - a front end that encapsulates dependencies with object dependencies. When creating a API or interface to my objects, i don't need to share all the details about how things work, just how to use it. The facade is a happy public face that hides the real complexity of the object.
Data Access pattern - data access objects like DAOs only know about the persistence layer or database. The data access pattern often includes helper methods to simplify the way the service layer see it and calls it.
SOA architecture
Here is the wikepedia definition of SOA. These principles are relevant to OO design in general. Try to build your CFC's in such a way that they can be used by more than just the application you are currently working on. Its easy to develop tunnel vision and only think about the task at hand when building and creating CFCs. It may be a little easier to think only in the moment, but that is junior league.
Think instead of who else could use this object and if someone else where to use it, what do they have to know about it. Think of the object as a small part of a larger system or environment. Think ecosystem and one plant or animal in that system. With that mind set, consider these principles in more detail.
Encapsulate - which means hide the complexity of its inner workings from the rest of the ecosystem.
- Think black box, others do not need to know how it works, just that it does.
- Think happy face to the rest of the world where the face is the API or interface. The API should show all interested parties how to interact with itself, what questions it can ask of it or what services it provides to the ecosystem and what information must be provided for the request to be understood and carried out by the object.
- Try to keep the knowledge of the environment out of the data access objects and only operate on data passed into it, do NOT rely on data that exists somewhere else, like in application or session scope. No reaching out.
Loose Coupling - This means minimum dependencies on other things in the environment. Let each thing be self sufficient.
- Objects in general have inherent dependencies, especially data service objects like DAOs and Gateways. Try to minimize these dependencies and even put them into a single place, like a object management service, like ColdSpring. At least then the coupling is managed and defined all in a single place and does not leak into the service layer.
Contract - This is an agreement that exists between two objects that interact.
- Think of this as a handshake or agreement that defines how communication occurs. This contract exist between two objects and should not change without informing both parties.
Abstraction - objects hold logic that is not exposed - only a contract is required to use it.
- Object a says to object b, i would like to call your method "get widget" where you require me to provide a widgetID. Object B says ok object a, you asked a question that i understand and provided me with the correct ID, so here it is - don't worry, i wont tell you about any of the gory details about how i completed this request for you. What? you want to know what else i can do for you? Ok, here is my API.
Re-usability - a holy grain of software design.
- Write code once and try to use it as much as possible. Some objects are inherently not too re-usable, like listeners and view files and controllers - they are all part of a specific application.
- The service layer and it dependent DAOs and Gateways are better candidates for reuse (IMHO). So, try to keep the knowledge of the environment out of the data access objects and only operate on data passed into it, do NOT rely on data that exists somewhere else, like in application or session scope. No reaching out.
Composition - collection of like services.
- This is the has a relationship that is referred to in UMLing. This is also how many APIs are wired together, via packages. Packages are like minded services.
Cohesion- a service, object or method is good at doing one thing.
- Do not try to be the doing of all things. Think in terms of small and numerous, not few and beefy. This is applicable to object design and method design.
Discover ability - the service should be able to be found.
- Especially if it is a public service, it should be findable. Using the hint property in CF is a good way to expose the intention of the object.
Wednesday, May 7, 2008
Object differentiation
There are two types of objects that we model in our CFCs.
1 - Value objects (beans)
Things, nouns (person, place or thing) that make up our domain.
Value objects or beans are the traditional objects that represent things in your domain. For example, one of our applications is called DataMentor, these things comprise its domain,
- assessments
- resources
- users
- standards
- institutions
- scores
- examples
Typically, each value object has a service object that is referenced by the application who requests to communicate with it. Remember that value objects are not very social, they only like to be spoken too by their own service. The service is the more social of the two, it likes to talk to its own value bean but also to other services too.
Value objects have 'state' which means they change within a session (the time a person spends using our application). In our application, we use state assessments. If a teacher is looking at different assessments, each time one is chosen, its 'state' should change, the details of the assessment would come from the database and the getter methods for each of the attributes would be called, updating the state of the object.
In our application, a user chooses an institution, a request is made to the database to get the details of the institution. Then the institution service is called to update the state of the institution value object (bean). The service calls the DAO which calls the setter methods for the institution value bean. This process of getting and updating institution details via services, DAO and value beans is repeated whenever a new institution is selected.
2 - Service or Infrastructure
These objects are singletons and have no state. They exist to serve the value objects. They are created once and do not change during the user session, used per-application. They may cache data, because once its state is created, it does not change, requests to refresh itself do not occur.
These objects live to serve the value objects in step 1. If a value object needs to be written to a database or refreshed from a database, the value object speaks to the service layer which speaks to the data access layer which speaks to the database.
Service layer objects are social, they like to speak to other objects, like listeners, controllers, data access models and other services.
There are inherent dependencies between the value and service objects. A value object needs a service object, a DAO needs a service object and the DAO object may need a data source object. These dependencies are difficult to manage, a object management framework like coldSpring can help. A common problem is tight coupling between the objects which means knowledge of each other is required to operate.
Often we use design patterns like Factory Pattern to solve common problems like this.
The Factory Pattern
A factory makes things, like widgets, wadgets and objects. The factory knows about dependencies and manages them. The objects in your application DO NOT need to know about each other. The factory creates the objects and the dependency objects. This pattern reduces coupling between dependent objects.
The question arises, who's job is it to know about the other. For example, who's job is it to ensure that when a service calls a DAO, that the DAO object exist? Typically, you see code in the constructor of the service that creates the object to the dependent DAO. This builds coupling between the two and forces the service to maintain application state, which is a best practice no no. Since the service object has a natural relationship with its DAO or Gateway, a better example might be the service needing information from another object, not its own. How does the service know if that object exists? Whose job is it to know this? Let the factory pattern implemented in a framework like ColdSpring manage the dependency and object creation.
coldspring
Following OO best practices like loose coupling (minimal dependency among interacting objects), promotes object creation and managing of its own state. In other words, I better be creating a object pointer to other objects that I depend on. The classic case is a DAO or Gateway data access object that has a service that manages it. The data access object has a dependency to the service layer. Without the its service, the application could never use the data access object. This dependency is often managed in the constructor method of the service. When the constructor method is called, object pointers are created to the data access layer objects.
If your application consists of a lot of objects, then lots of these type of dependencies are created and lots of object instantiation calls are made. Enter coldSpring.
Coldspring is itself a framework that plugs into another framework, like mach II.
ColdSpring helps to manage all this object instantiation and dependency stuff. The creation and dependency relationships are removed from the service layer and abstracted away into a coldSpring configuration file, that is registered in the mach ii controller file.
Read this article and follow others from this blog.
machii framework
Mach II uses a XML based notation that uses a central controller as the road map for the application. The controller (/config/mach-ii.xml) holds definitions to other parts of the application, like listeners, filters, plug ins and view files. The main workhorse of the controller is the event. The application has lots of events, each of which contains all the "work" needed to complete the event request. Each event is like a small little autonomous part of the application. Events lend themselves to web development because almost every action taken in a GUI can be thought of as an event, like clicking a button or choosing from a list or clicking a link. The directions needed to complete the request are encapsulated in the mach II event.
The mach II controller file has listeners, filters and plugins registered as part of its configuration. The Listener, filter and plugin registrations are conceptual pointers to CFC files in the application. Best practices suggest that the listeners be registered to Services.
Filters and plugins are special types of CFCs that can be registered within specific events to automate frequently needed code, like login check.
The controller also has a properties section that contains hard coded values that are used and referenced throughout the application.
If the event requested needs data then a listener notification is made and a service manager is called, possibly with some arguments or parameters (data needed to process the request). The service or manager would call into the data access layer which would then in turn call to the persistence layer or the database. In many cases, the data is then returned from the database to the data access object to the service back to the controller who stored the result in an event argument (variable) that is then consumed or displayed by the presentation layer or view file.
MachII implements this type of MVC design patten and is specifically designed to work best with well defined objects that adhere to these principles.
Event arguments are event specific or scoped variables defined in the event that are available to the listeners and view pages to help control flow or make decisions during execution of the event.
software design ecosystem
I like the ecosystem analogy that shows the relationship of the parts of a software system or application. The ecosystem has discrete parts that function independently from the system, but also contribute to the overall health of the entire system. Each role plays its part and contributes to the general health of the application.
In the software ecosystem, there are parts such as a database (persistence layer), the model(data layer), with its data access objects like DAOs and Gateways and their services, the controller (road map to the application and manages requests and directs program flow) and the presentation layer (view files or GUIs).
Specifically - the controller has registrations or pointers to the presentation files and to the listeners that then point into the model layer. The listeners speak to the model layer via a service layer. The service layer is not really part of the application aside from being registered in the controller. The service layer should be application agnostic (like Sgt Shultz of Hogens Heroes fame - "i know nothing"). If the service layer only worries about the data layer beneath it, and not about maintaining state for the application, it is much more reusable to other applications and follows design best practices by modeling loose coupling.
This tiered ecosystem promotes individual health in each discrete part but also promotes the overall health of the system. Healthy organs = healthy body.
Internalizing - software design
Design patterns
Once you start to build objects you see commonality among them and the need to predict or manage their interactions. Design patterns help the architect to build predictable manageable scalable solutions. In other words, use patterns to help build great flexible code.
Tuesday, May 6, 2008
design patterns
First thanks to Kurt W, Peter and Matt Woodward for the workshop in machii. Also, Sean Corfield for the excellent description and illustration around design patterns.
Why design patterns?
Promote tiered development, which promotes the separation of parts of the application. The bus. logic is separated from the database logic which is separated from the presentation logic. This makes the application easier to troubleshoot, support, add new things to etc.
A very common design pattern is MVC
Programmers can work on different areas of the same application at the same time. One programmer can work on the bus. logic while another is working the presentation layer while a third is working in the database.
Less experienced programmers will typically not see the value in the tiered approach to software design. They will put the logic or code into a single of set of files, grouping together database, business and presentation layer code or logic. This approach is adequate if the application is a one time shot, never to be added to or updated again. If the application grows, the code becomes more difficult to maintain and troubleshoot as new code and logic is added. If a new programmer comes on to maintain, more difficulties arise because there are no standards followed.
More experienced programmers see the pay off and need for a well tiered application. As new requirements come along and new programmers are introduced to the application, code separation and predictability become more valued.