Finding Tables for Employee Census Report

This is one of the posts where I describe the more technical aspects of a project I've been working on. The contents here are meant mainly to trigger my own memory when needed in the future.

The HR department has requested a census report so that the company can shop around for benefits with other vendors. In typical fashion, the information needs to be laid out in such a way that none of the "canned" reports from the business system will suffice.

I know where to get the demographic information (employee ID, client ID, city, state, zip, salary, etc.), but this is the first time I've had to track down information such as premium rates, employer contribution, employee contribution, and coverage tier.

I sat with the HR director to find out where she's able to see this information. Back at my desk, I launched a SQL Server Profiler Trace, filtering to my login ID and session ID. Then I navigated to those same screens to help identify which objects in SQL Server I need to look at.

From there I've been able to target the specific tables I need for my SQL scripts.

-----
Check out my other blogs:
Journey Inside My Mind Blog
Journey Inside My Mind Podcast
Get That Job!
QuotesBlog
Twitter.com/danieljohnsonjr

Related tags: daniel+johnson+jr

Reorganizing clients in Darwin provides opportunity for SQL Server clean-up

This is a post where I share more of the technical aspects of my job as a 'conscientious programmer/analyst'.

The company I work for is reorganizing clients into new databases in the Darwin business system (a customized version of Microsft Great Plains Dynamics), and the business sees this as a great opportunity to clean up a lot of things. This view is especially shared by us who work in the IT department.

The 12 current databases are, for the most part, the same in their structure; i.e., same tables, views, stored procedures, and so forth. The company has used these multiple SQL Server databases to for specific types of clients, based on their industry classifications, etc.

Street Sweeper,
originally uploaded by itsray.

One bit similarity is in the paycodes that are used. Paycodes, for the purposes of what I refer to in this and other posts on this blog, refer to specific codes that are used to signify specific payroll transactions. The company has paycodes set up for regular and overtime hours, commissions, bonuses, mileage reimbursements, and other types of income. Additionally, there are codes for deductions, such as cash advances, uniforms, payments made by the employee for benefits and 401(k). Moreover , there are codes set up for benefits, which include an employer's contributions to health care and 401(k), and the like. Finally, there are a separate set of codes for both state taxes and local taxes.

It may be easy to imagine, then, over time, and as clients come and go, that the databases would have lots of various codes. Mirror that across 12 databases, and it becomes more complicated. Furthermore, I've discovered that the code descriptions are not consistent from database to database. That the business has a need to reorganize clients into new databases presents a great opportunity to clean things up, as a result.

Yesterday, the Director of IT and the Director of Special Projects asked for a list of active codes for active employees, across all 12 databases. I am the guy they turn to in order to get this done quickly. Because of my experience with how the databases have been set up, I usually know pretty quickly which tables to use in my SQL scripts.

In this particular case, I was interested in the Transaction History table, since it contains the three most important elements my internal customers needed: check date, transaction type, and paycode.

I initially set up the script to pull all paycodes, but I found close to 10,000 codes in use since the business started using Darwin in 2005. I checked with the Director of Special Projects, and she asked me to limit to just those codes in use since October 2007. Thankfully, that narrowed the list to just under 2000. I also included, at her request, the name of the database in which the codes were used. This proved especially helpful, since not all codes are in use in all the databases.

On my way home last night, I called into Jott to remind myself to set this up as a stored procedure.

Just another way I'm able to help keep the business engine going.

-----
Check out my other blogs:
Journey Inside My Mind Blog
Journey Inside My Mind Podcast
Get That Job!
QuotesBlog
Twitter.com/danieljohnsonjr
Utterz by danieljohnsonjr

Related tags: daniel+johnson+jr, sql+server, programming, business+intelligence, microsoft+great+plains, payroll, peo, darwin

Keeping upper management knowledgeable and salespeople paid

This is another post where I share technical details about a project I have been working on.

SITUATION

Sales executives within the company receive monthly commission checks based on active client employee counts and gross payroll, for clients that they have brought on. In addition, upper management needs to see high-level numbers such as active clients, active employee counts, and gross payrolls - dashboard-type information.

A easy-to-use tool to generate this information did not exist at a user level. Previously, upper management relied on IT or the Controller to generate this information and send it to them.

Through some personnel reorganization, the process for generating this report fell through the cracks. Salespeople were waiting for their commission checks for the previous month, so the project was both urgent and important.

As usual, this information needs to come from the multiple SQL Server databases the company uses to manage client information through the Darwin PEO System, a customized, version of Microsoft Great Plains for the Professional Employer Organization (PEO) industry.

TASK

I was asked to develop a tool that upper management can use to generate information themselves. Some of the application requirements and thoughts that guided the development:

• Let users pick the date range, click a button, and have the system produce a report.
• Develop the application quickly to meet the immediate needs of the organization, yet with the ability to be reused whenever upper management so desires.
• Since upper management is most comfortable with Microsoft Excel and will want the data in a workbook anyway, use Excel Visual Basic for Applications (VBA)) and ActiveX Data Objects (ADO) within a single Excel workbook to produce the results.
• Choose Excel over Access because the application overhead is low (i.e., no need for tables, forms, reports, etc.).
• Since the company doesn't mark employees and clients as inactive in the system immediately when they are terminated, define an active employee during a date range as a paid employee.
• In addition to a paid employee count, obtain a total check count and gross payroll amount for each client during the date range.
• If an employee received a check and it wasn't voided, it counts.
• Take advantage of server-side processing to achieve the best performance.

ACTIONS TAKEN

I first developed the SQL statement to unite data across the twelve SQL databases, based on prior knowledge of where to find information. Then I wrapped the SQL statement up in a stored procedure, with start and end dates as parameters.

After testing the procedure with different date ranges to make sure the information was accurate and made sense, I moved on to the Excel piece. I wrote code in Excel VBA and ADO to execute the stored procedure and output the results to a worksheet in the workbook.

Once I had tweaked the completed application to make sure everything ran smoothly, I e-mailed it to the director who requested it.

RESULTS

Within a few minutes I received a phone call from her, telling me how awesome I am. She also sent the application to the owner of the company so that he can run the report as often as he wants.

Now they are able to generate the information in a matter of seconds themselves, versus waiting for the Controller or someone else in IT to generate it for them; or, even worse, spend hours compiling the information themselves.

-----
Check out my other blogs:
Journey Inside My Mind Blog
Journey Inside My Mind Podcast
Get That Job!
QuotesBlog
Twitter.com/danieljohnsonjr

Related tags: daniel+johnson+jr business+intelligence sql+programming microsoft+great+plains data+mining

Reaching the point of diminishing returns on a project

Over the past month, I have been working on a project to automate payroll entry for a client that has close to 300 employees and is growing. Naturally, it is the company's best interest to get this payroll automated as much as possible.

To do this I've been developing an application to bridge the payroll info to Microsoft Great Plains Integration Manager.

One of the criteria I've used to evaluate a client's payroll is a consistent layout in a good format, so that I can tell the program where to map regular hours, overtime, cash reimbursements, etc. In some cases I'm able to tweak the payroll to make it into a layout that is consistent enough for me to automate.

This particular client is a construction company, and the only electronic version of the payroll is a text file that is not delimited. The file is basically a report from the client's system, and it has page headers and department footers, as well has breakdowns of withholding, etc. Most of that I can ignore in the program.

My colleague was able to import it into Microsoft Excel, using spaces as the delimiter. From there I saved the file as tab-delimited text, the format used in the other bridge applications I've developed.

The big problem is the inconsistent layout, which has come from using spaces to delimit the text, depending largely on how much detail is on a particular line of text. Look at the following 3 lines of text:

• E 22 Per Diem - C AZ 3 99
• E 22 Per Diem - C AZ 3 XYZ111 99
• E 22 Per Diem - C 99

In each of these lines, the payroll item is Per Deim, and the amount is 99. When you space-delimit the lines the amounts are in different columns.

This is just one major aspect of complexity that has come from my attempt to automate the payroll. After talking it over with my boss, we realize that we've come to the point of diminishing returns.

I'm off this project until there is another way to parse the payroll information and am able to move on to another project that's in the queue.

Related tags: microsoft+great+plains integration+manager payroll+automation microsoft+access vba programming application+design peo daniel+johnson+jr

Every programmer should be on a testing project

I think every programmer should get a chance to be on a testing project because it will make him or her a better programmer. On one project I supported the complete software development lifecycle process to develop a multi-tiered, Intranet-based, Product Data Management System from a software quality assurance perspective. The company is a global consumer products company, and they were creating a system to manage information about their products.

To do this the testing team, of which I was a part, analyzed business requirements and functional specifications, and designed test cases. Then we built test data and executed the test cases. For the most part, each of us would concentrate on a specific aspect of the system.

Since my time on this and other testing projects, I've found myself approaching development from the perspective as a tester, and it's helped me be a better programmer. I wish the same for every programmer.

Helping company take advantage of a tax credit

I found out last year from the Accounting department at my current company that employers with tipped employees can get a tax credit based on how employees have reported their tips. I guess it's the IRS's way to encourage employers to get their employees to report tips so that they can get the tax from it.

The PEO model is one of co-employment: the client employs someone to do the work while the PEO employs the worker from an HR perspective. If the client doesn't take the tipped employee tax credit (or whatever it's actually called), then the PEO can take it.

For me it has meant developing yet another report from the SQL Server database. The business, of course, wants the report as soon as they can get it, and I and my boss help them understand that it cannot always happen overnight or even in a week.

The report, as many of these I develop lately, involves looking at the transaction history on employees' paychecks to identify reported tips. It sounds simple, and it really is, as long as you know where to look.

Over the past year or so, I've gotten better as I've come to understand where the types of information are stored - the database and the front-end are a modified version of Microsoft Great Plains, and the tables are not always intuitively-named.

I probably should push this application out to the Accounting department, since all I did was run it for last year and the current year. The upper management was certainly surprised at fast I'd turned their request around.

Programming for Good to Reduce Necessary Evil

A few weeks ago I mentioned a project I'm working on to streamline the benefits auditing reports. This process involves verifying premium payments for health insurance with what is taken out of employees' paychecks. As I mentioned in the previous post (via the Twitter photo, no doubt), the application I'm working on will save 2-3 weeks' worth of work and many trees' lives.

I just met with the HR department to discuss the status, and the HR Specialist who does this work mentioned that the current process wastes a lot of time. But needed by the business, I said.

"It's a necessary evil," the HR Manager replied.

"Well, hopefully we can make it for something good," I said.

Programming for Good to Reduce Necessary Evil. That's what I'm about.

Migrating Existing Access Applications to Access 2007

This post is one of the more esoteric ones where I delve into the geeky details of some of my programming work. I know - it's really sexy, isn't it?

I have mentioned the bridge application I developed that helps make payrolls run faster, helping client employees get paid faster, and so forth, using Microsoft Access 2003 with VBA, ADO, Excel, Office, etc.

Some members of the company are starting to migrate to Office 2007, and we can see the entire organization moving there soon. A few weeks ago, I tried opening and running one of the bridge applications in Access 2007, and it bombed horribly, specifically in how I've written it to use the Office 11 FileSearch object.

I just found a couple of few documents on MSDN and TechNet that I hope will help understand what is involved in the migration:

• Transitioning Your Existing Access Applications to Access 2007
• Migration Considerations for Access 2007
• Changes in Access 2007
• Office 2007 Object Model Reference
• Migrating a VBA Solution to a Visual Studio Tools for Office Add-In

Okay, to be honest, I picked that last item because it sounds interesting.

More Benefits Auditing and Follow me on Twitter

I'm going to put up a Twitter badge in the sidebar so you can know about other stuff I'm doing. For now, feel free to follow me on Twitter, and let me know you found me via this website.

Tags: daniel+johnson+jr twitter benefits+auditing analysis

Handling errors in the payroll bridge application

Clients email the company their payroll files, and the columns on the payroll indicate Regular Hours, Overtime, Vacation, and so forth. To make the bridge application work, I and the other developer map the program that assigns payroll items based on this layout. It only works, however, when the layout is consistent. Whenever a client changes it by adding or deleting a column, the program generates an error.

One thing I have found so helpful in resolving errors is the Error Handler function. I use the Err object in Visual Basic, but I also add additional information such as the module name and procedure name where the error occurred. The user gets a message box, which is helpful for them, but I also wrote the program to create a log file with the same information so that I can go back and see errors that have occurred.

Rapid response to payroll layout change

One hour ago I got an e-mail from a payroll specialist, telling me that a client with 107 active employees had added a column to their payroll file and asking if I could fix their bridge application to allow for the new payroll item.

I opened the code, and after I made the changes needed in a few class modules, we were good to go. I tested it out on my machine and then put it out on the network for the payroll specialist to use.

I was able to get this done so quickly because there were only a few places to add some more code. Having the system set up like this was very efficient.

Expanding the 401(k) reporting tool

As the company grows, the need to be able to report on client's using the 401(k) plan does as well. We are adding clients to additional databases, and that has meant that the 401(k) reporting tool needed to be updated so that the user can identify which clients are participating in the company plan.

Using Microsoft SQL Server Management Studio, I created the additional tables: one for each additional database. Then I added ODBC links to these tables in the 401(k) reporting tool, the desktop application created using Microsoft Access 2003. I have to remind myself that, for each computer using this tool, I have to create the ODBC connection to each database and table. Then I updated the VBA code so that it can recognize and use the new tables as it compiles the information for the reports.

After testing out the changes, everything appeared to be working fine. I contacted the user and updated her version of the tool, refreshing the ODBC links where needed.

The company is now more ready to begin managing the 401(k) program for new clients.

Learning .NET

I'm a little jazzed to say that I'm finally starting to learn .NET, since that's what recruiters tell me all the cool kids know. I'm going to focus on VB.NET, since that's the most easily transferable skill from what I already know.

It looks like I'm not alone: LinkedIn: Answers: What Microsoft Visual Basic and SQL Query Language Resources would you recommend?

When the feature became a problem

When I was initially developing the bridge application a couple years ago, I had set up the Close button on several forms to be the Cancel button. That way a user can just hit the Esc button to close the form, if they prefer to use the keyboard.

As the applications for various clients have been used, this feature had become more of a bane than a benefit. It turns out that users hit the Esc key to Undo what they have typed, but the application would close the file. This results in some records being incomplete, causing errors when the application is processing.

While you can try to anticipate the user's experience as much as possible beforehand, some bugs still show up.

No big deal. I just removed that functionality from every bridge application we have in use so that program behaves in a way that users expect it to behave.

Lesson learned while on a mission critical conversion project

Here's a project I worked on where I learned an important lesson.

The client was a global consumer products company, and the project was to convert an Access 2.0 application to Access 95, which was a conversion from a 16-bit environment to a 32-bit environment. I'm not even going to claim that I understand what all that means, except to say that there were a lot of procedure calls to the Windows Application Programming Interface (API) that needed to be changed over. This application was used to facilitate preparing the company's profit/loss and balance sheets for all of their business units around the world. Hereafter, I'll call it the "financial reporting application".

Where I went wrong was that I didn't test portions of the program as I converted the code. It demonstrated a skill in programming that I needed to learn, and I learned that lesson the hard way because, once the program was "converted", i.e., all the code compiled correctly with no errors, there were other application errors that began popping up all over.

I created a tool to help me keep track of all the errors and the steps taken to resolve them and provided the client with daily progress reports.

As I think back on it, I would have done much better if I had taken more time to understand how the program worked in its prior environment and tested the program incrementally as I revised the code.

Deploying a much-needed search tool in the workplace

We have a limited number of licenses to use the payroll management software in our organization. As we have added more staff whose responsibilities involve using that software, those licenses get used up pretty quickly. It's not uncommon to receive 5 emails every day asking if someone would log out of the system so that someone else can get in to search something or run a payroll.

With some knowledge of the database, we've been able to develop a tool to work around the system by querying the server directly. In fact, this is exactly what I've done when developing the 401(k) reporting tool and other custom tools for the organization.

We have created a search tool that members of the organization can use to return some basic information about an employee. I may have mentioned before that we have several databases for different clients, and many times, we don't know the for whom the client works or in which database his or her records are. As a result, the user has to use trial and error to find the employee, which is time-consuming.

Our new Search Tool, which is what I'll refer to it as here, returns an employee's address and company name, along with whether he or she is marked as Active in the system. I have to give a lot of credit to my fellow developer for taking the intial idea I had, and, with some guidance on the SQL, has created the tool to execute the search.

I spent part of the day deploying the tool to different members of the organization who would most likely use it the most: One person from the Risk Management department who coordinates the paperwork related to unemployment claims; one person who is fielding calls from employees about not getting W-2 forms (probably because of an address change); and the receptionist, who helps out with overflow work, especially with respect to returned mail.

We're still building on to the tool while the application is in use, but we are developing in a separate environment. Once again, by having this application on a shared network drive, we can update the file on that one location as needed.

Backing up the bridge applications and drive-independence

When I was developing the bridge applications, I had set the code up to recognize a "test" environment and a "production" environment through hard-coded filenames. It became cumbersome to change the filenames in code every time I moved the application to each environment. We also realized we needed to set up a "standalone" environment for the applications.

...hence, the need for the application to be made "drive-independent".

I wound up setting a property in one of the main VBA classes to "auto-discover" its environment as the application launched. That made it possible to move copies of the application easily among the test/development, production, and standalone environments.

While we're on that topic, every week we need to backup the copies of the bridge applications from the production environment to the other two. The reason for this is because there are certain changes made to some self-correcting features while payroll specialists run the applications each week. (I'll have to discuss these features in a separate post.)

For a few weeks I would manually navigate to each folder that contains a bridge application and copy the application file over to the other environments. That's okay for a couple of files, but as we started implementing this for several clients, it became very cumbersome.

The solution: batch files. I created batch files that copy all the files from the production environment to the development/test and standalone environments. What used to take 45 minutes now takes 1 minute. Way cool.

Mining data from 3,000 separate Excel files

One of the consulting projects I worked on was for the finished product exports department at a global consumer products company. The company produces products that get shipped all around the world, but this particular group concentrated on North America.

Because of the North American Free Trade Agreement (NAFTA), the company was able to save millions of dollars each year by taking advantage of tariff exemptions for products shipped among the U.S., Canada, and Mexico. Prior to my joining the project, the company had discovered from their legal department that they needed to provide documentation that these products were indeed exempt.

The regulations describe raw materials that compose the finished products, so we needed to look at the constituent level for every product the company manufactures. I was informed that the company had close to 3,000 Excel spreadsheets for various products that listed the composition of each. Additional information would be forthcoming in the near future after I had started on the project.

I looked at the Excel files and discovered that the composition was not in a flat layout (headings and rows), and thus, I wouldn't readily be able to import the information into a database table. I would need to manipulate the data into such a layout and then import it into a database.

Instead of going brute force into all 3000 of these files, I opted to automate the process. I decided to build a conversion tool in Excel that would search for the relevant contents of each file and copy and paste them into a new spreadsheet in a table layout. I wrote VBA code in a standalone Excel workbook to do the job. In addition, I also wrote code to record errors and other anomalies in a log.

I also wrote some VBA code in Microsoft Access to launch the Data Conversion tool in Excel, along with some use of the Microsoft Office object interface. I put all the directories where the files were located into a Lookup table and had the program go through each directory, performing the conversion on each Excel file, importing each Excel spreadsheet's information to a master database table.

I wound up being able to click a button and watch as the program performed the conversion. I had set up a flag in the Lookup table to indicate whether files were imported successfully. Then I would check the log in Excel for anomalies, handle them appropriately and retry.

I'll share more about this project in future updates.

Fixing the client's 401(k) reporting application to make it run better

The company didn't have a process in place to make sure that this one client receives their 401(k) information on a regular basis. I had developed an application last fall for this specific client to produce the information, but there was no one set up specifically to run the application on a regular basis.

I was contacted earlier today to set up this application on several people's computers. Interestingly enough, everyone could get to the application, since it exists on a networked drive, but, to make things as easily accessible as possible, I prefer to put a shortcut to the application on users' desktops.

After I finished that, I sent an email to everyone affected, letting them know about the application, and letting them know how to get the report.

About 15 minutes later, I was summoned to a user's computer because there was a problem with the report details. She was running the report for last week, but the Last Pay Date for several employees was for this week.

As I looked into the program, I realized that all the other details were correct except for this Last Pay Date, which gets updated whenever a payroll is run. I began brainstorming other ways to get the correct date.

One option was to look at the last pay check date for the date range specified, which is from the first of the month to the date the user selects on a form. That would make sure that date for employees who are getting paid this week is the date the user selected.

As I thought about that, I realized I was not finished. What if we run this report a month from now, and an employee had been terminated this month? The client would still want that employee to show up on the list.

I continued brainstorming, and wrote some mock VBA code on my marker board. Satisfied with that, I created a function, tested it out with various scenarios, and eventually came to a solution that will either use the last pay date for the reporting range OR the Last Pay Date field from the other data table.

I generated the report and sent it off to the client service representative, who will then forward it on to the client.

Revised payroll layout forces changes to bridge application

In an earlier post I described some changes to a client's payroll journal. In this one I'll share how I worked those changes into the corresponding bridge application we use to automate the payroll entry.

You will recall that I made two significant changes:

1. Added a new payroll item and
2. Removed totals columns

I will discuss each of these in turn, along with some other things I've discovered while working on this application.

New Payroll Item

The client wanted to be able to track Training, so I added it to their payroll journals. In the bridge application, I added the payroll item in the respective classes to recognize it.

Totals Columns

The client tracks both weeks of their pay period individually, and the payroll journal was originally set up to provide totals for each week and for both weeks. The totals columns for Week 1 were set up immediately after Week 1's payroll items. The totals columns for Week 2 and for both weeks were set up after Week 2's payroll items.

When I removed the totals columns, I moved all of Week 2's payroll items. The bridge application contains a VBA class where I've mapped the payroll item columns on the spreadsheet to elements of a tab-delimited array. I revised the code to make sure the each element is mapped appropriately.

Pitfalls while working off a network

Early Friday afternoon the network where I develop the applications became frozen, and I had to shut down the application from the Task Manager. Knowing this corrupts the application file, when I came in this morning, I started over again, but this time off a local copy, in case something would happen to the network again. Since I'd made the changes one time already, it didn't take long to do.

Testing it like the user would

When I tested the application to make sure the changes worked, the output looked odd. I contacted the payroll specialist and found out that she runs the bridge application for one location at a time, and after testing the application that way, everything worked out fine.