Tuesday, September 25, 2007

Proclarity

Proclarity analytics now a key component of Microsoft BI stack, combining the strengths of Proclarity data analytics with Microsoft Reporting Services, Business Scorecard Manager and SharePoint Portal Server the product stack provides a robust BI solution.

Architecture










Proclarity for Reporting Services Proclarity for Reporting Services makes creating OLAP reports easier than ever with a wizard-driven user interface. Proclarity authors can create and publish reports using the Reporting Services XML Report Definition Language (RDL) and export WYSIWHYG views of data directly to an RDL file and that can be deployed on the report server. Report authors can conduct additional formatting or programming on datasets created with Proclarity.

As well the reporting services reports can also be easily integrated into a proclarity dashboard. The Proclarity dashboard has the capability to pass the parameters between the analytic views and reporting services.

Key Benefits:
  • Reduced time and effort - Proclarity wizard-driven UI makes it easy to create reports or new OLAP data sets in Reporting Services without having to type complex MDX queries by hand
  • Lower Cost of Ownership - The seamless integration of Proclarity and Microsoft Reporting Services leverages current technology investment at a fraction of the cost of competing solutions
  • Reduce the time to make an informed decision - Quickly linking between Reporting Services reports and Proclarity Analytics allows users to easily transition between monitoring and analyzing

Proclarity for Business Scorecard Manager Proclarity analytics are made available in the Business Scorecard Manager(BSM) through a feature called "report view." This allows decision makers to discover the exceptions and analyze the root cause while doing the performance monitoring.

With Proclarity report view within a scorecard, one can easily move from monitoring performance to full ad-hoc analysis. This powerful connection leads to better decision making and a more successful overall scorecard solution.

Key Benefits:

  • Zero-footprint, thin-client, Web-based analytics
  • Seamless analysis of what's driving the KPIs
  • Web-optimized charting and advanced visualizations
  • A seamless link to additional ad-hoc investigation

With Office PerformancePoint Server 2007 V1 release the scorecard/dashboard features will be available as part of the PPS.

Proclarity for SharePoint Portal Server Microsoft SharePoint Portal Server and Proclarity for SharePoint Portal Server offer a robust portal solution with embedded analytics that provide at-a-glance views of targeted KPIs and important business views. These elements work together to enable faster and more informed decision making. Together, Proclarity and Microsoft continue to gain strength in the business intelligence (BI) industry.

Proclarity for SharePoint Portal Server is a Web part that displays Proclarity analytic views within a SharePoint Portal system. It enables the deployment of a zero-footprint Web client with centrally managed business logic, making it easy for IT staff, business users and analysts to publish meaningful analytic views to a SharePoint Portal Server with zero code.

Key Benefits:

Reduce the time and effort to embed analytics into the portal environment.

  • Seamless integration - easily create and add Proclarity analytic views into SharePoint Portal
  • No custom development required

Easily deliver tailored analytics within portal

  • Easily create Web parts to display Proclarity views in a SharePoint Portal Server
  • Put as much analytic power into portal - from static views to full ad hoc interactivity
  • Link the full Proclarity analytic client to any Proclarity view within the portal enabling a quick transition to powerful analytic capabilities

One version of the truth in portal environment

  • Information and business logic are centralized to provide decision makers with access to consistent data and common definitions
  • Publish views and metrics based on predefined business rules
  • Easily consumed KPIs are quick to add and maintain
  • Best analytic practices within company are now easily shared through the portal

Monday, July 9, 2007

BlackPearl – K2.Net

The latest model of K2.Net is the BlackPearl. The upcoming release of K2 “BlackPearl” includes a set of tools that enable a new generation of users to collaboratively assemble dynamic business applications from reusable items.

The K2.net “BlackPearl” platform provides for full life-cycle analysis and development of your business solutions.Its unique modeling experience supports multiple, interchangeable modeling canvases (Process Design, role-based, documentation-based, and custom) and modeling tools (Office-style Windows application, Ajax-based web application, Visual Studio 2005 and Microsoft Visio 2007).K2.net “BlackPearl” ensures that the core process definition is the same regardless of the canvas or tool used to model the process by abstracting the view information from the process definition.The result is a solution platform that allows each participant to have the process modeling experience, regardless of the role (business user, developer, administrator, etc), to use the modeling canvas and tool with which they are most comfortable.

K2 “BlackPearl” will also include the following

  • K2 “BlackPearl” Event Bus
  • Simulation Modules
  • ADO.NET Data Provider for K2 "Black Pearl"
  • New data level security modules
  • Process Inheritance
  • Visual Process Debugging
  • Visual Data Mapping
Description:

The K2.net “BlackPearl” process modeling life cycle will likely involve multiple participants using multiple modeling canvases from multiple modeling tools while all remaining in sync and producing a common process definition output.The K2.net “BlackPearl” Studio environment allows the designer to model the process using the view paradigm that works for them.K2.net “BlackPearl” will provide an impressive array of designers for developers, business analysts and all designers in between.

K2 is still a workflow product (now built on top of Microsoft’s Workflow Foundation), but it also becomes a true BPM platform, something the old version was not. K2 is also strong as an Enterprise Application Integration platform (leveraging the new BizTalk/.NET Adapter Framework built on Windows Communication Framework). K2 is also an Application Server with strong support for rapid application development (leveraging SOA standards and principles, SQL Server, the .Net Framework, Visual Studio, and the Office Platform). So it is a complete application server and development platform with a strong workflow engine at its core, and a very flexible integration platform built-in.

K2 BlackPearl provides multiple design tools that allow all stakeholders to actively participate in the creation of dynamic business applications.Any one can participate in the design and creation of processes,reports,forms and policies and use what’s been built to rapidly assemble applications to help them drive their business.

Thursday, July 5, 2007

Maverick.NET Framework

Maverick.NET is a .NET port of Maverick, a Model-View-Controller framework for web publishing. It is a minimalist framework which focuses solely on MVC logic, allowing you to generate presentation using a variety of templating and transformation technologies

Features :
• Pluggable view templating technologies.
• Pluggable transformation technologies.
• Support for iterative transformations.
• Configuration using an XML sitemap.
• Framework support for internationalization, customization of content based on browser, and WML.
• The ability to use standard ASPX pages as Maverick controllers. There is full support for Server Controls, Postback forms, and View State.
• A fairly exhaustive set of sample applications using a variety of templating languages and controller types.

Depending on what templating technology you choose, you may be interested in one or more of the following features:

• Automatic XML serialization of arbitrary objects so that XSLT transformations can be used without the effort and processing overhead of generating and parsing text XML.
• The ability to halt iterative transformations at any step and obtain the intermediate result. In the case of XSLT and DVSL, this would produce static XML. This allows designers to build template files offline with simple tools.
• "Wrapping" transformations using basic templating languages like ASPX. This allows varying content to be easily encapsualted within a common layout, look-and-feel, etc.
• NFop transformations which can produce PDF documents on-the-fly.
• NVelocity and DVSL (an XSLT-like language based on Velocity) templates are now supported through the Opt-NVelocity package.

Working :
There are two basic phases in the life of an executing Maverick application. The first is the "load" phase which (usually) occurs once and uses the config file to construct a tree of workflow objects. The second is the "execution" phase in which http requests are serviced. Maverick is designed to perform as much work as possible during the load phase so that the execution phase can be as fast as possible.

The workflow tree constructed during the load phase represents all of the possible execution paths for servicing an http request. It consists of objects which implement various interfaces, and many of the objects are created by pluggable factories. After the workflow tree is built, the overhead Maverick itself adds to a running web application should be little more than a couple IDictionary lookups and a handful of virtual method calls.

The top level object is an ICommand object, which corresponds to a single defined command in the config file. Other than special implementations like the reload command, there are two basic implementations of the ICommand interface: CommandSingleView and CommandMultipleViews. The CommandFactory will automatically build the correct instance depending on how many views are available; the only difference is that CommandSingleView can eschew the IDictionary lookup to deterimine which view to render.

There is always one IController associated with every command; a simple null controller is generated by the ControllerFactory when no controller is specified by the user. In addition, it should be noted that from the perspective of the command objects, controllers always look like singleton controllers; a special ThrowawayAdapter is used for "normal" controllers.

The command object chooses an IView to render. IViews are built by the flexible IViewFactory system. At runtime, execution passes from the ICommand to the IView identified by the IController.Go() return value; the IView is responsible for "dealing with" the request from here out. There are many types of views.
If an IShuntFactory was defined in the modules part of the config file, the IView instances held by a ICommand will actually be instances of the decorator ViewShunted, which holds an IDictionary of real views and adds the mode switching behavior.

Actual IView implementations are responsible for tasks like sending HTTP redirects, or rendering the model using an ASPX page. Output is rendered through an ITransformStep object obtained from the IViewContext passed into Go(). Depending on the configuration in maverick.config, the ITransformStep may be the entry point for a sequence of transformations or it may actually dump the results directly to the response output stream.

Thus, at execution time, an HTTP request is serviced like this:
• The Dispatcher receives the request and looks in an IDictionary for the ICommand associated with the particular URL. ICommand.go() is called.
• The ICommand calls Controller.Go(). The controller optionally sets the model object (using a method on IControllerContext) and returns a string indicating the name of the view to render.
• The ICommand looks in an IDictionary for the IView to render. IView.go() is called and passed a IViewContxt object.
• If shunting was enabled, the IView will actually be an instance of ViewShunted. This object uses the IShunt to find the actual view to render; the actual mechanism for this is specific to the shunt.
• The IView generates some content and sends it to the ITransformStep obtained from IViewContext.NextStep. If there were no transforms defined, this step will actually be a special implementation that sends output directly to the real response.
• ITransformStep objects are chained together such that one passes information to the next. Eventually the data gets passed to the special implementation which sends output to the real response.

Thursday, June 21, 2007

Single Sign-On

A gud link to know about Single sign-on

http://www.opengroup.org/security/sso/sso_intro.htm

Tuesday, June 19, 2007

InRule - Rule Engine

InRule is built entirely on the .NET platform, leveraging a component architecture that delivers unparalleled rule processing performance and flexibility. From the feature rich rule authoring environment to the optimized execution of the business rule engine, InRule unifies the domains of managed business logic and traditional systems processes.














Description:

InRule's new visual rule flow and modeling provides a graphical view of rule application elements and how they are related. Visual rule flow gives InRule users greater ability to understand where a rule application starts and ends and gives them a method for visually following the way one or more rulesets execute. Users can view and define rules through the point and click interface. Pan and zoom functionality makes it easy to navigate within large or complex applications.
















Features :

  • InRule's catalog services provide the ability to store, version, check-in/check-out, and set permissions on rules.

  • Administrators can now view all the objects stored across multiple catalog instances and can "promote" a rule application from one catalog to another.

  • InRule can be used as a Web-based Rule Management with AJAX Functionality
    Business Language Authoring and decision table controls are implemented as a set of embeddable web controls, providing greater flexibility in determining where rules can be maintained and edited

  • Testing rule logic has been made even easier with irVerity, InRule's integrated testing component. Rule execution is simpler to follow with an improved layout and Watch Lists.

  • Users can now define rules that are applied depending on the context in which a field is used.

  • InRule's support for handling collections has been enhanced with new functions for effectively iterating over a series of items in a collection

  • InRule allows users to specify rule sets to activate or deactivate based on category or metadata.

  • User can choose to activate only online rules and deactivate batch rules depending on their needs

  • A new Halt Ruleset function gives rule authors the capability of halting the execution of a rule with or without invoking a runtime error

Monday, June 18, 2007

Xml Web Services in Sql Server 2005

Database products are making it easier and easier to hook up Internet protocols directly to your data. Microsoft SQL Server is a case in point. While Microsoft SQL Server 2000 did allow Web data access, the process required to use the SQLXML library and a full installation of Internet Information Services. With SQL Server 2005, Microsoft has solved this issue.Now, you can have direct access to your SQL Server data from any HTTP SOAP client without any extra middleware at all - not even IIS.

Description:

Microsoft continues to bet heavily on Web Services as a backbone for service-oriented architectures, so it makes sense that they've chosen to expose SQL Server 2005 data via a Web Service. Or, to be more precise, you can create as many Native XML Web Services as you like in SQL Server 2005.

To do so, you use the new CREATE ENDPOINT statement to create HTTP endpoints. Each HTTP endpoint ties the results of a SQL Server stored procedure directly to a Web Service, providing support through the Web Services Description Language (WSDL) and Simple Object Access Protocol (SOAP) protocols for retrieving the data supplied by the stored procedure. SQL Server interfaces directly with the Windows HTTP listener process (http.sys) so that SOAP requests are routed directly to SQL Server, with no intervening middleware necessary. Similarly, SOAP responses are sent directly back to the requesting client.

Steps:
1)Create a simple Stored Procedure

CREATE PROC dbo.SalesStoreProc AS
SELECT ProductID, ProductName FROM Sales.Product

2)Creating an HTTP Endpoint

The next step is to create the HTTP endpoint. Running this SQL statement is what makes the data from the stored procedure available to SOAP clients:

CREATE ENDPOINT GetStores STATE = STARTED
AS HTTP( PATH = '/Store', AUTHENTICATION = (INTEGRATED),
PORTS = (CLEAR),
SITE = 'localhost')
FOR SOAP
( WEBMETHOD 'StoreList'
(NAME='AdventureWorks.dbo.SalesStoreProc'),
BATCHES = DISABLED,
WSDL = DEFAULT,
DATABASE = 'AdventureWorks',
NAMESPACE = 'http://AdventureWorks/Store')
GO

Key things to be noted:

  • The STATE clause specifies the initial state of the endpoint. It can be started, stopped (listening but returning errors to clients) or disabled (not even listening for requests)
  • The AS HTTP clause specifies the transport protocol to use. You can also specify AS TCP here.
  • The PATH clause specifies the URL on the server that clients will use to reach this Web service.
  • The AUTHENTICATION clause specifies how clients will authenticate themselves to the SQL Server: BASIC, DIGEST, NTLM, KERBEROS, or INTEGRATED.
  • The PORTS clause specifies whether the service will listen on the CLEAR or SSL ports, or both (other clauses, not shown here, let you specify non-standard port numbers)
  • The SITE clause lets you specify a hostname for the computer that will respond to requests.
  • The FOR SOAP clause states that this endpoint will respond to SOAP messages. Other endpoints handle messages for Service Broker or database mirroring.
  • The WEBMETHOD clause defines a Web method, mapping a method name to the name of a stored procedure
  • The BATCHES clause specifies that this endpoint won't process arbitrary SQL statements.
  • The WSDL clause specifies that it will provide WSDL support.
  • The DATABASE clause specifies the database that contains the data.
  • The NAMESPACE clause specifies the XML namespace for the messages.


Testing the Webservice:

The URL for the WSDL file (http://localhost/Store?wsdl) is determined by the CREATE ENDPOINT statement. The Web server is the server where SQLServer 2005 is installed. The PATH clause dictates the Store portion of the URL. The remainder of the URL is where SQL Server listens, by convention, for WSDL requests associated with this particular Web Service.
Thus SQL Server will accept, and respond to, standard HTTP SOAP 1.2 requests. This gives you a supremely easy way to hook SQL Server 2005 data into any sort of service-oriented architecture that you can dream up.

Wednesday, June 13, 2007

Microsoft Windows Small Business Server 2003

Microsoft Windows Small Business Server 2003 is the easy way for small business customers to get the power of the Windows Server operating system plus Microsoft server-based solutions for e-mail, fax, database, and security-enhanced, shared Internet access, and file/print/calendar/application sharing-all in one integrated, simplified solution at an affordable price.Windows Small Business Server 2003 helps your small business customers work smarter with fewer resources.




















Microsoft Windows Small Business Server 2003 will have the following installed
 Microsoft Windows Server 2003
 Microsoft Windows SharePoint Services V2.0
 Microsoft Exchange Server 2003
 Outlook 2003
 Microsoft Shared Fax Service
 Microsoft ISA Server 2000
 Microsoft SQL Server 2000
 Microsoft Office FrontPage 2003
 ROUTING AND REMOTE ACCESS SERVICE(RRAS)
 Mobile User/Device Support
 Shared Network Resources
 Backup and Restore
 Task Based Management

System Requirements:

• 300-megahertz (MHz) Processor
• 256 megabytes (MB) of RAM
• 4 gigabytes* (GB) of available hard disk space

Features :

• Protect key business information with an infrastructure that includes built-in firewall protection and security-enhanced remote access to help prevent unauthorized users from getting into your network.
• Keep your PCs and servers current with the latest software updates to enhance the security of your network.
• Prevent data loss by automatically backing up company information and enabling employees to retrieve accidentally deleted files and restore earlier versions of files
• Store, find and share information in one centralized location
• Provide employees with an internal Web site so they can find and share files and collaborate on group projects
• Work from virtually anywhere with remote access to business information and resources – e-mail, calendars, network files, internal Web sites and business applications
• Share resources and equipment such as Internet access, printers and fax machines
• Stay connected to your customers from virtually anywhere and anytime with remote access to business information and resources
• Manage your customer relationships more effectively with a centralized place to store and exchange information
• Use Windows Mobile-based devices to access e-mail, contacts, and calendars from virtually anywhere and at anytime
• Reduce costs and maximize limited resources

Monday, May 7, 2007

Merging Pdf

There is a gud site about Dynamic pdfs and merging 2 pdfs.

http://www.cete.com/Products/MergerForNET/

Sunday, May 6, 2007

Microsoft Identity Integration Server 2003

Microsoft Identity Integration Server 2003 (MIIS 2003) provides services that make it possible to manage identity-related data from multiple, unrelated data sources. The goal of MIIS 2003 is to provide organizations with a unified view of all known identity information about users, applications, and network resources. MIIS 2003 enables you to synchronize identity information across a wide variety of identity stores, easily provision and de-provision accounts across systems, and enables self-service and helpdesk-initiated password management and reset from a Web browser. This improves productivity, reduces security risk, and reduces the total cost of ownership associated with managing and integrating identity information across the enterprise.

Microsoft® Identity Integration Server 2003 is a service that stores and coordinates identity information from multiple sources (for example, directories, databases, or formatted dump files) in an organization. With Microsoft Identity Integration Server 2003, you can combine that information into a single logical view that represents all of the identity information for a given user or resource.

An important element of MIIS 2003 is a namespace called a metadirectory. A metadirectory can integrate and synchronize information that is stored in multiple data sources including directories.

MIIS 2003 uses the metadirectory to process identity information from different data repositories such as the Microsoft® Active Directory™ directory service, a Microsoft SQL Server database, IBM® Lotus® Notes, Novell® eDirectory™, or a fixed-width text file. Every data store that provides well-known data access methods is a potential data source candidate for MIIS 2003. The different kinds of identity data systems used by MIIS 2003 are called connected data sources.

The MIIS 2003 metadirectory uses two namespaces to perform its tasks. Namespaces in this case are database-like structures containing uniquely named objects that store information during the synchronization process. The two namespaces are the connector space and the metaverse (MV), as shown in Figure 1.


The connector space is a storage area where object additions, deletions, and modifications are written before they are synchronized with the metaverse or the connected data source. A portion of the connector space is dedicated to each management agent. It is important to note that the connector space does not contain the connected data source object itself, but a shadow copy of the object that contains a subset of the object's attributes, as defined in the management agent. Not every object in a connected data source is designated for synchronization; for example, inactive employees might not be designated. For objects that are designated, some attributes might not be included because some objects contain numerous attributes that are not used at all or that might not make sense in the data sources with which the data is being synchronized.

The connection with the connected data source is not maintained continuously but only connected during communication sessions to update the connected data source or the connector space.

Management agents control the data flow between a connected data source and the metaverse. There is a management agent for each supported connected data source. The metaverse is another core component of MIIS 2003. It is a storage area that contains the aggregated identity information from multiple connected data sources. It provides a single, global, integrated view of all combined objects. These metaverse objects are based on the identity information that is retrieved from the connected data sources and a set of synchronization rules that specify how MIIS 2003 creates the metaverse objects. Metaverse objects are used to capture the aggregated data during synchronization so that data can flow to the other data source in the synchronization design.

MIIS 2003 uses a connector space and the metaverse to create an aggregated view of the identity data that is stored in multiple repositories. You can use this aggregated view to synchronize identity information across applications and platforms.
For more information about the metadirectory and concepts associated with MIIS 2003, see “Essential Concepts of Microsoft Identity Integration Server 2003” in the Technical Reference collection of the MIIS 2003 Technical Library.

Monday, March 5, 2007

Microsoft Office InfoPath 2007

InfoPath is based on Extensible Markup Language (XML). When you design a form template, InfoPath creates an .xsn file, which is a cabinet (.cab) file that contains the files necessary for the form to function, such as XML Schema Definition(XSD) and XSL Transformation (XSLT) files. When a user fills out a form in InfoPath, the data in that form is saved or submitted as industry-standard XML. However, you don't have to know anything about XML to design a form template or fill out a form. The point is that XML can make it easier for your organization to repurpose the data that it collects by using forms. For example, a single InfoPath form template for trip reports can be used to provide XML data to a customer relationship management system, a petty cash system, and a travel planning system.

Description:
InfoPath can be used to collect business data from the people whom you work with, including your colleagues, partners, suppliers, and customers. For example, you can use InfoPath to create a form template for expense reports for the people in your organization.
InfoPath form templates can be straightforward, simple form templates that are used by several people in a small workgroup. For example, a 10-person sales team can use an InfoPath form to informally collect and share information about sales calls. The data in those forms can be merged into a single summary report that is sent to management each month.
Alternatively, organizations can design highly sophisticated form templates that are connected to existing corporate databases or integrated into existing business systems. For example, the developers in your information technology department can design an InfoPath form template to manage the expense reporting process for your organization. The form template can include views and business logic features that enable different categories of users to submit the expense report, review it, approve it, and reimburse the submitter.

Usage
InfoPath can be integrated with
a) Microsoft Office Outlook
b) Share Point Server
c) Biztalk Server
d) WebServices
e) K2.Net etc.,

Monday, February 5, 2007

What is there in .Net 3.0?

.NET 3.0 includes:

  • .NET Framework 2.0:
  • WinFX:
  • InfoCard:

One conclusion you should draw from this is that .NET 3.0 is really .NET 2.0 plus more functionality. In other words, the existing .NET 2.0 Framework and compilers are unchanged, so your existing code will continue to run as it exists today. All that .NET 3.0 does is add new functionality in four key areas.

Windows Communication Foundation is essentially the next generation of Web Services (asmx), Web Services Extensions (WSE), .NET Remoting, Enterprise Services, and Microsoft Message Queue (MSMQ). This is a lot to absorb, and it is important to recognize two key things. First, none of these existing technologies is going away anytime soon; WCF is merely another alternative to them. Second, in its first release, WCF often uses these existing technologies behind the scenes. WCF is a new API to access various concepts, but it doesn't necessarily replace the existing technologies, and it sometimes consumes them.If you build distributed systems using n-tier, client/server, or service-oriented architectures, then WCF will be of interest to you.

Windows Presentation Foundation is the first major new graphic UI framework to come out of Microsoft in more than a decade. At its core, WPF is the replacement for CGI and CGI+, which are the graphics foundation for the entire Windows platform. At a higher level, WPF is an object-oriented framework on which you can build forms-based 2D and 3D interfaces. In the short term, WPF is an alternative to DHTML and JavaScript for Web development and to Windows Forms for Windows development. In the long term, WPF is the logical successor to both Windows Forms and any sort of interactive Web development.If you build rich, interactive user interfaces and find DHTML, Ajax, or Windows Forms limiting from a graphics perspective, then WPF will be of interest to you.

Windows Workflow Foundation is often characterized as being a subset of BizTalk Orchestration that you can host within your own process. It is a workflow engine that runs within your Windows client, Windows server, or ASP.NET Web application, allowing you to use a visual designer to define how your various workflow tasks interact. If you have ever thought that using a workflow or a state machine would be useful, but BizTalk Server seemed like overkill, then this technology should be of great interest to you.

Windows CardSpace, which was code-named InfoCard, is Microsoft's implementation of a standards-based solution for a global identity management technology for the Internet. In other words, WCS is a way of managing your identities on the Internet in a way that is easier and more secure than today's reliance on usernames and passwords. And unlike previous attempts at a unified sign-on, such as Microsoft Passport, with WCS the credentials are managed entirely by the user and the application with which they are connecting.

For example, you might have a casual identity for blogging or posting on Web forums, in which case you'll manage your credentials directly. But you might also do online banking, in which case your bank will likely manage your banking credentials. In any case, Microsoft isn't managing your credentials.

WCS is integrated directly into Windows Vista, so this technology promises to become important to any of us who develop or use Web applications.The .NET Framework 3.0 is unique in that it represents a major set of functionality in .NET, and yet doesn't change any existing .NET Framework 2.0 libraries, compilers, or functionality. This new framework represents the future of .NET development, and whether you have immediate need for the new functionality or not, it is important to be aware of these new features because they will define the shape of applications in the long run.

Sunday, January 7, 2007

Improve ADO.Net performance

Here are a few tips to improve ADO.Net performance

Poor performance is frustrating to an end user, and can lead to users not using your applications in the intended manner. Take advantage of these five tips to accelerate the performance

Although IT organizations invest significant resources in optimizing the network topology and database design of their applications, many IT organizations overlook the performance aspects of the database middleware. Developers overlook the impact that the ADO.NET provider has on the application, even though a significant percentage of the response time is related to the time spent requesting and receiving data from the database.

Performance and scalability problems can be debilitating to the overall success of an application and ultimately to the success of the development team. If an application suffers from poor response time, user productivity suffers, service-level agreements are violated, and the reputation of the development organization is maligned. For critical systems, application performance issues can be tied directly to business success in the form of increased cost, decreased revenue, and assumption of additional risk.

Some organizations deal with performance and scalability issues in a reactive fashion, because they lack the development and testing procedures necessary to optimize the application. They simply develop the application and deal with performance issues as they arise in the development environment.

A much better solution is to attempt to take the requisite steps to make sure your application performs to your user’s expectations, both from a functional and performance standpoint. However, there are quite a few steps you can take on your own to make sure your application processes task in the most efficient manner before production deployment.

One of the key reasons for user performance complaints in database apps is that developing .NET data access code that performs fast isn’t easy. The ADO.NET documentation includes only basic guidelines and interface definitions to help programmers develop apps using ADO.NET, and it provide next to nothing in terms of prescriptive guidance to developers who want to write code that performs well. There is very little guidance for writing code that performs well.

That said, you’re not entirely on your own. You can take quite a few steps that will result in code that performs faster and more reliably. I’ll walk you through several of the common performance pitfalls that I see made on a regular basis, as well as how to avoid them.

Tips: 1

Fast to Code != Fast Code

Many programmers use the DbCommandBuilder object because it can save time when coding a new application that uses DataSets. However, this shortcut can have a negative effect on performance. Built-in concurrency restrictions can lead to the DbCommandBuilder generating highly inefficient SQL statements. For example, suppose you have an eight-column table called EMP that contains employee records. The DbCommandBuilder object generates this UPDATE statement:

CommandText: "UPDATE EMP SET EMPNO = ?, ENAME = ?, JOB = ?, MGR = ?, HIREDATE = ?, SAL = ?, COMM = ?, DEPT = ? WHERE ( (EMPNO = ?) AND (ENAME = ?) AND (JOB = ?) AND ((MGR IS NULL AND ? IS NULL) OR (MGR = ?)) AND (HIREDATE = ?) AND (SAL = ?) AND ((COMM IS NULL AND ? IS NULL) OR (COMM = ?)) AND (DEPT = ?) )"

You can write much more efficient UPDATE and DELETE statements than the ones the DbCommandBuilder generates. For example, assume you’re working with the previous example, and you know the underlying database schema. Also, assume that you know the EMPNO column of the EMP table is the primary key for the table. You can create a much simpler UPDATE statement that retrieves the same results:

UPDATE EMP SET EMPNO = ?, ENAME = ?, JOB = ?, MGR = ?, HIREDATE = ?, SAL = ?, COMM = ?, DEPT = ? WHERE EMPNO = ?

This statement runs much more efficiently on the database server than the statement the DbCommandBuilder generated.

Another drawback of the DbCommandBuilder object—it generates statements at runtime. Each time a DataAdapter.Update method is called, the DbCommandBuilder analyzes the contents of the result set and generates UPDATE, INSERT, and DELETE statements for the DataAdapter. The programmer can avoid this extra processing time by specifying the UPDATE, INSERT, and DELETE statements for the DataAdapter explicitly.

Tips: 2
Avoid retrieving long data if you don’t need it. Retrieving long data across a network is slow and resource-intensive. Remember that when you use a DataSet, all data is retrieved from the data source, even if you never use it. However, some applications don’t formulate the select list before sending the query to the .NET data provider. In other words, some applications use this syntax to accomplish sending the query:

send SELECT * from ..
If the select list contains long data, most .NET data providers must retrieve that data at fetch time, even if the application never binds the long data result columns to display to the user. You should try to implement a method that limits the number of columns you retrieve whenever possible.

Users Don’t Want Long Data

It also helps to remember that most users don’t want to see long data. If the user does want to process these result items, the application can query the database again, specifying only the long columns in the select list. This method allows the average user to retrieve the result set without paying a high performance penalty for network traffic. Consider this query:

SELECT * from EMPLOYEES WHERE SSID = '999-99-2222'

An application might want to retrieve only this employee’s name and address. Unfortunately, a .NET data provider doesn’t know which result columns an application wants to retrieve when the query is executed. A data provider knows only that an application can request any of the result columns. When the .NET data provider processes the fetch request, it will most likely return one or more result rows across the network from the database server. In this case, a result row contains all the column values for each row, including an employee photograph if the Employees table contains such a column. Limiting the select list to contain only the columns you need results in decreased network traffic and a faster performing query at runtime.

Tips: 3
Another common performance pitfall concerns how you handle commits. Committing transactions is slow because of disk I/O and, potentially, network I/O. Always start a transaction after connecting; otherwise, you remain in Autocommit mode.

A commit involves several actions. The database server must flush back to disk every data page that contains updated or new data. This is usually a sequential write to a journal file. By default, Autocommit is on when connecting to a data source, and Autocommit mode usually impairs performance because of the amount of disk I/O needed to commit every operation.

Also, some database servers do not provide an Autocommit mode natively. For this type of server, the .NET data provider must issue a COMMIT statement explicitly and a BEGIN TRANSACTION for every operation sent to the server. You also pay a performance penalty for up to three network requests for every statement issued by an application—in addition to the large amount of disk I/O required to support Autocommit mode.

Consider this code fragment that starts an Oracle transaction:

DbProviderFactoryf = DbProviderFactories.GetFactory( "DDTek.Oracle");
try {
conn = f.CreateConnection();
conn.ConnectionString = ("Connection String info");
conn.Open();
DbTransaction transId = conn.BeginTransaction();
DbCommand cmd = conn.CreateCommand();
cmd.CommandText = "select * from users";
DbTransaction transId = conn.BeginTransaction();
cmd.Transaction = transId;
DbDataReader reader = cmd.ExecuteReader();
// Continue to work with transaction boundary.
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}

This approach can make an enormous difference. I once had a customer who performed 5,000,000 inserts to DB2. He heard me give this tip during a talk and made the change to his application. The insert went from taking five hours to taking ten minutes!

Using transactions can help application performance tremendously, but don’t take this technique too far. Leaving transactions active can reduce throughput by holding locks on rows for long times, preventing other users from accessing the rows. You should commit transactions in intervals that allow maximum concurrency.

Tips: 4

Use DbCommand.Prepare() Appropriately

Using the DbCommand.Prepare method can have a significant positive (or negative) effect on query execution performance. The DbCommand.Prepare method tells the underlying data provider to optimize for multiple executions of statements that use parameter markers. Note that you can Prepare any command regardless of the execution method used (ExecuteReader, ExecuteNonQuery, or ExecuteScalar) .

Consider a .NET data provider that implements DbCommand.Prepare by creating a stored procedure on the server that contains the prepared statement. Creating stored procedures involves substantial overhead, but you can execute the statement multiple times. Doing so minimizes the cost of executing that statement because the query is parsed and optimization paths are stored at create procedure time. Applications that execute the same statement multiples times can benefit greatly from calling DbCommand.Prepare and then executing that Command as needed.

However, using DbCommand.Prepare for a statement that is executed only once results in unnecessary overhead. Furthermore, applications that use DbCommand.Prepare for large, single-execution query batches exhibit poor performance. Similarly, applications that either always use DbCommand.Prepare, or never use DbCommand.Prepare, do not perform as well as those that use a logical combination of prepared and unprepared statements.

Tips: 5
Much has been written about when to choose DataReaders over DataSets. However, it is a critical choice when it comes to performance, so I will add to the significant amount of advice that is already there.

Let me sum it up as bluntly as I can: The DataReader will always be faster at fetching data. Period.

The DataSet uses a DataAdapter to retrieve the data from the database. The DataAdapter uses the DbDataReader when it is reading data. Given this, you might wonder why the DataReader is always faster. The answer lies in the fact that DataSet performs additional processing once the data is fetched from the DataReader, converting the data to the internal format of the DataSet and storing it in memory.

Once it has the data in memory, the DataSet maintains both the original and any changed data, leading to even higher memory usage. This can also lead to a scalability problem, depending on the size and number of copies that you keep around.

All that said, the DataSet is much more functional. It allows for random fetching (the DataReader is forward-only), it gives XML capabilities to relational data, it is disconnected from the database so it does not use up resources on the server, and it is updateable. My bottom line recommendation for performance is to use a DataReader when you don’t need the additional functionality of the DataSet.

Each one of the tips I’ve mentioned can improve the performance of your applications. Taken as a whole, these tips can have a considerable impact, not just on raw application speed, but in user satisfaction and in the ability of your company to meet its business goals.

Friday, January 5, 2007

DBA recommendations for writing Stored Procedure

1. Try to use derived tables when possible, instead of temporary tables to gain better performance.
ex:
SELECT MIN(Salary)
FROM (SELECT TOP 5 Salary FROM Employees ORDER BY Salary Desc)

2. Do not use 'SELECT * from table'. Only return the columns you want/need. This brings more data back over the network and also eliminates the use of some indexes (covering indexes). This may also screw up apps that are expecting a certain number of columns when a column is added.

3. Use the @@error global variable for error-handling, but store the value into your own user variable, as the value will be reset after the next statement. All Global variables should be stored into a local variable if they are going to be used later in processing.

4. Always 'SET NOCOUNT ON' at the beginning of stored procedures. This will reduce the network round trips your SQL will have to make.

5. Avoid using 'print' in SPs unless it is needed. Print statements cause network trips and acknowledgments from the client. If the client does not need to know - or nothing is receiving the 'print' - this is unneeded.

6. Develop and stick with a standard naming convention for your area/project.

7. Never use a wildcard character (%) at the beginning of a search string as it will always result in a table scan.

8. When doing an insert in an SP - Always specify the column names - to avoid issues when a column is added to the table.

9. ALWAYS specify the 2 part name when executing SPs.
ex: EXEC dbo.sp_proc. When the 2 part name is not used SQL Server first checks for an SP owned by the executer, then dbo.

10. Keep Transact-SQL transactions as short as possible within a stored procedure. This helps to reduce the number of locks.

11. Don't use the prefix "sp_" in a stored procedure name. The reason for this is that whenever a stored procedure is executed with the prefix of 'sp_' SQL Server first looks in the MASTER database to execute it - if it does not find it in MASTER it will then use the current database.

12. If you use input parameters in your stored procedures, you should validate all of them at the beginning of your stored procedure.

13. Avoid nesting transactions in stored procedures. If you need transaction support from SP to SP use a savepoint.

14. If your SP does not require 'transaction safety' use the 'NOLOCK' table hint. A table hint lets the SQL engine ignore and not perform locks for a given operation.

15. Try to avoid using temporary tables inside your stored procedures. Use Table Variables if possible. Temp Tables Reduce the chance of plan reuse. Table variables also have less locking overhead, making them faster.

16. Place all DDL language at the top of the Stored Procedure. All DML should follow. This will increase the chance of plan reuse.

17. Use the 'TOP' operator over the 'SET ROWCOUNT' command to limit the number of rows returned, as there is less overhead. (And ROWCOUNT is going away after SQL2K).

18. Use Global temp tables only if you absolutely have to.

19. Clearly document and mark any optimizer hints.

20. Limit your joins to 4 tables or less. The SQL Server optimizer is only so smart (and so is your DBA).

21.Do not use a function in the where clause.
For example if you are looking for records with a timestamp greater
than 7 days old don't say
where ts > getdate() - 7.
Set a variable "SELECT @date = getdate() -7" and then use:
Where ts > @date

22. Do not use ‘Where not exists’.
Instead use a left outer join where the left side is null

Thursday, December 28, 2006

Advantages of INSTEAD OF Triggers

You can write a trigger for a view, but if the view is updateable it isn't necessary. Triggers on the underlying table fire automatically. (Of course, you may have your own reasons why you want triggers on such views.) Of all the advantages INSTEAD OF triggers offer, the main one is that they allow views that would normally not be updateable to support updates. A view that involves multiple tables must use an INSTEAD OF trigger to support inserts, updates, and deletes that reference data in more than one table. For example, you can write an INSTEAD OF trigger that inserts rows in multiple tables from a single view.

Another important advantage to INSTEAD OF triggers is that they allow you to write logic that accepts parts of a batch while rejecting other parts. Finally, INSTEAD OF triggers allow you to take some alternative action in the event of some particular condition that the application defines as an error.

Thursday, December 21, 2006

Isolation level in SQL

Transaction Isolation Levels

Closely tied in with the modes and methods of locking is the transaction isolation level. To understand the new locking behavior, you need to understand the four transaction isolation levels in SQL Server 7.0: Uncommitted Read (also called "dirty read"), Committed Read, Repeatable Read, and Serializable.

IsolationLevels
The isolation level that your transaction runs in determines how sensitive your application is to changes other users' transactions make, and consequently, how long your transaction must hold locks to protect against these changes. The ANSI SQL standard defines four levels of transaction isolation. Although previous versions of SQL Server let you specify all four distinct levels of transaction isolation, there were only three different behaviors because SQL Server internally treated two of the syntactic specifications (i.e., Repeatable Read and Serializable) as synonymous.

You can change the level of isolation that a particular connection is operating in by using the SET TRANSACTION ISOLATION LEVEL command. Keep in mind that the SET command applies only to your current connection, and every time you make a new connection (or open a new window in the Query Analyzer), you'll be back in the default isolation level. I'll use each of the four isolation levels in the examples to follow.

To see how each level behaves, you can use the script in Listing 1, page 20, to create a table with a few rows in it. I'll refer back to this table in examples for each of the four isolation levels.

UncommittedRead
Uncommitted Read, or dirty read, lets a transaction read any data currently on a data page, whether or not that data has been committed. For example, although another user might have a transaction in progress that has updated data, and that transaction is holding exclusive locks on the data, your transaction can read the data anyway, and possibly take further actions based on the values you read. The other user might then decide to roll back his or her transaction, so logically, those changes never occurred. Although this scenario isn't desirable, with Uncommitted Read you won't get stuck waiting for a lock, nor will your reads acquire share locks that might affect others.

Let's see how Uncommitted Read behaves. Use the SQL Server 7.0 Query Analyzer, and start two separate connections. Use the pubs database in each one. In the first connection, begin a transaction, but don't commit it:

BEGIN TRAN
UPDATE ISOLATION_TEST
SET col2 = 'New Value'

Now, use the second connection, and change your isolation level before trying to access the same table.
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
SELECT * FROM ISOLATION_TEST

All the values in col1 are 0, even though the transaction in the first connection has not committed yet. In fact, the transaction might never commit. If you took some action based on the fact that all the values are 0, you could regret it if the changes turned out not to be permanent. Back in the first connection, roll back the transaction:

ROLLBACK TRAN

Now rerun the SELECT statement in the second connection to see that all the values are back to what they were before. If you're following along with these examples, make sure you close your connections after each one, so that all outstanding locks are released.

CommittedRead
Committed Read is SQL Server's default isolation level. It ensures that an operation will never read data another application has changed but not yet committed. Because you can never read uncommitted data, if a transaction running with Committed Read isolation revisits data, that data might have changed, or new rows might appear that meet the criteria of the original query. Rows that appear in this way are called phantoms.

So Committed Read behavior has two aspects. To see the first aspect, you can run the above example, without setting the second connection to use isolation level Read Uncommitted. The second connect would then block on the SELECT statement; it can't read the changes the first connection has made but not yet committed (or rolled back). To see the second Committed Read behavior, close all the connections in the Query Analyzer from the previous example, and open two new connections using pubs again. In the first connection, run the following batch:

SET TRANSACTION ISOLATION LEVEL READ COMMITTED
BEGIN TRAN
SELECT AVG(col1) from ISOLATION_TEST

In the second connection, update the table:
UPDATE ISOLATION_TEST
SET col1 = 500 WHERE col1 = 50

Notice that the update is successful, even though the first connection is still inside a transaction.
Go back to the first connection and run the same SELECT statement:

SELECT AVG(col1) from ISOLATION_TEST

The average value is now different. The default isolation level does not prevent another connection from changing data you have read. Because you are not guaranteed to see the same data if you rerun the SELECT within the transaction, the read operations are not guaranteed to be repeatable.

RepeatableRead
If you want the read operations to be repeatable, choose the third isolation level. The Repeatable Read isolation level adds to the properties of Committed Read by ensuring that if a transaction revisits data or if a query is reissued, the data will not have changed. In other words, issuing the same query twice within a transaction won't pick up any changes to data values that another user's transaction has made. No other user can modify the data that your transaction visits as long as you have not yet committed or rolled back your transaction.

To see Repeatable Read behavior, close all the connections, and open two new ones in pubs. Issue the same two queries as above, but this time, have the first connection

SET ISOLATION LEVEL REPEATABLE READ.

The second connection will have to use a slightly different update statement, because the value of 50 for col1 no longer exists:

UPDATE ISOLATION_TEST
SET col1 = 5000 WHERE col1 = 500

This update will block when it tries to update the ISOLATION_TEST table. And the first connection will get the same result when it reissues its original SELECT. Preventing nonrepeatable reads is a desirable safeguard, but it comes at a price. The cost of this extra safeguard is that all the shared locks in a transaction must be held until the completion (COMMIT or ROLLBACK) of the transaction.

However, Repeatable Read isolation doesn't prevent all possible changes. It protects only the data that you have read. The following example shows you what this protection means. Close all connections, and open two new ones connecting to pubs. In the first connection, start a transaction in Repeatable Read isolation level and look for all rows that meet a certain condition.

SET TRANSACTION ISOLATION LEVEL REPEATABLE READ
BEGIN TRAN
SELECT * FROM ISOLATION_TEST
WHERE col1 BETWEEN 20 AND 40

In the second connection, insert a new row:
INSERT INTO ISOLATION_TEST
VALUES (25, 'New Row')

Go back to the first connection, and reexecute the SELECT:
SELECT * FROM ISOLATION_TEST
WHERE col1 BETWEEN 20 AND 40

The second time you execute the same statement, the new row appears. Because the row doesn't even exist the first time you run the SELECT statement, it isn't locked. This new row that appears is called a phantom. You can prevent phantoms with the fourth isolation level.

Serializable
The Serializable isolation level ensures that if a query is reissued, no data will have changed and no new rows will appear in the interim. In other words, you won't see phantoms if the same query is issued twice within a transaction. Rerun the example from the Repeatable Reads section, inserting a row with a col1 value of 35. But this time, set your isolation level to SERIALIZABLE. The second connection will block when you try to do the INSERT, and the first connection will read exactly the same rows each time.

You pay a price to prevent phantoms. In addition to locking all the data you have read, enforcing the Serializable isolation level requires that SQL Server also lock data that doesn't exist! The Serializable level gets its name from the fact that running multiple serializable transactions at the same time is the equivalent of running them one at a time—that is, serially—regardless of sequence.

Controlling the Isolation Level SQL Server's default isolation level is Committed Read, but as you've seen, you can override this setting within your application. The most straightforward way is by using the SET command:

SET TRANSACTION ISOLATION LEVEL
[READ UNCOMMITTED READ COMMITTED REPEATABLE
READ SERIALIZABLE]

Previous versions of SQL Server treated Repeatable Read and Serializable as synonymous. I thought the difference was that Repeatable Reads prevented UPDATE operations, and Serializable prevented INSERTs and DELETEs. But the difference is in what data is locked. Repeatable Read locks only the data that has been read. With Serializable, SQL Server has to guarantee complete serializability, so it locks ranges of data.

Wednesday, November 15, 2006

Tips to choose the appropriate data types in SQL 2000

SQL Server 2000 stores data in a special structure called data pages that are 8Kb (8192 bytes) in size. Some space on the data pages is used to store system information, which leaves 8060 bytes to store user's data. So, if the table's row size is 4040 bytes, then only one row will be placed on each data page. If you can decrease the row size to 4030 bytes, you can store two rows within a single page because two rows can be placed into data page. The lesser the space used, the smaller the table and index, and lesser the I/O SQL Server has to perform when reading data pages from disk. So, you should design your tables in such a way as to maximize the number of rows that can fit into one data page. To maximize the number of rows that can fit into one data page, you should specify the narrowest columns you can. The narrower the columns are, the lesser the data that is stored, and the faster SQL Server is able to read and write data.

Try to use the following tips when choose the data types:

If you need to store integer data from 0 through 255, use tinyint data type.
The columns with tinyint data type use only one byte to store their values, in comparison with two bytes, four bytes and eight bytes used to store the columns with smallint, int and bigint data types accordingly. For example, if you design tables for a small company with 5-7 departments, you can create the departments table with the DepartmentID tinyint column to store the unique number of each department.

If you need to store integer data from -32,768 through 32,767, use smallint data type.
The columns with smallint data type use only two bytes to store their values, in comparison with four bytes and eight bytes used to store the columns with int and bigint data types accordingly. For example, if you design tables for a company with several hundred employees, you can create an employee table with the EmployeeID smallint column to store the unique number of each employee.

If you need to store integer data from -2,147,483,648 through 2,147,483,647, use int data type.
The columns with int data type use only four bytes to store their values, in comparison with eight bytes used to store the columns with bigint data types. For example, to design tables for a library with more than 32,767 books, create a books table
with a BookID int column to store the unique number of each book.

Use smallmoney data type instead of money data type, if you need to store monetary data values from 214,748.3648 through 214,748.3647.
The columns with smallmoney data type use only four bytes to store their values, in comparison with eight bytes used to store the columns with money data types. For example, if you need to store the monthly employee payments, it might be possible to use a column with the smallmoney data type instead of money data type.

Use smalldatetime data type instead of datetime data type, if you need
to store the date and time data from January 1, 1900 through June 6, 2079,
with accuracy to the minute.

The columns with smalldatetime data type use only four bytes to store their values, in comparison with eight bytes used to store the columns with datetime data types. For example, if you need to store the employee's hire date, you can use column with the smalldatetime data type instead of datetime data type.

Use varchar/nvarchar columns instead of text/ntext columns whenever possible.
Because SQL Server stores text/ntext columns on the Text/Image pages
separately from the other data, stored on the Data pages, it can
take more time to get the text/ntext values.

Use char/varchar columns instead of nchar/nvarchar if you do not need to store unicode data.
The char/varchar value uses only one byte to store one character,the nchar/nvarchar value uses two bytes to store one character,so the char/varchar columns use two times less space to store data in comparison with nchar/nvarchar columns.

Tuesday, October 3, 2006

Indexed view

An indexed view allows indexes to be created on views, where the result set of the view is stored and indexed in the database.

Indexed views work best when the underlying data is infrequently updated. The maintenance of an indexed view can be higher than the cost of maintaining a table index. If the underlying data is updated frequently, then the cost of maintaining the indexed view data may outweigh the performance benefits of using the indexed view.

Indexed views improve the performance of these types of queries:
  • Joins and aggregations that process many rows.
  • Join and aggregation operations that are frequently performed by many queries.
  • Views can be used to partition data across multiple databases or instances of Microsoft® SQL Server™ 2000.
  • Views in all versions of SQL Server are updatable (can be the target of UPDATE, DELETE, or INSERT statements), as long as the modification affects only one of the base tables referenced by the view.