Control-M/Server and Control-M/EM Hardware Sizing Template

As per BMC Documentation you can provide below details to BMC so they can suggest you hardware sizing.

All values must be maximum values including future planning.
Control-M/Enterprise Manager
New Installation (First time install of EM on site)
Number of Jobs defined for this EM:
Maximum Number of Active Jobs per Day (from Peak Usage Report) :
Maximum Number of Executions per Day (from Peak Usage Report) :
Maximum defined EM users:
Maximum concurrent EM users (logged in at the same time):
Database Type and Version (e.g. PostgreSQL, Oracle, Sybase, MSSQL):
Database configuration (local or remote)
Number of Datacenters (CTM/Servers) connect:
Is BIM/Forecasting/SelfService enabled:
If BIM is enabled how many Services are defined:
What is the average number of jobs per Service:
Is Forecast enabled:
Is Self Service enabled:
O.S. Platform and Version (e.g. HP 11.31, Solaris, AIX)
List of other applications/programs that will be residing on the machine
Intended Hardware to be used - if there is any (
Model:
Number of CPU’s:
Number of Cores/CPU:
Speed of CPU Mhz:
RAM:
Version of EM that is Intended to be Installed
If possible, Machine Spec according to the following website –
http://www.spec.org/cpu2000/results/rint2000.html
Upgrade from an Existing Installation
Is response time today on existing installation satisfactory?
How loaded is the machine today? (Measurement of CPU and Memory peek values)
Current Existing EM Version
Control-M/Server (For each Datacenter)
Maximum Daily Job Count
Maximum Daily Job Executions
Number of Agents connected to the server
How many of the Agents are using Agentless technology
Required Average Job Processing Rate for 15 Minutes (optional)
Required Job Submission Rate for 15 Minutes (optional)
O.S. Platform and Version (e.g. HP, Solaris, AIX)
Database Type and Version (e.g. PostgreSQL,Oracle, Sybase, MSSQL)
Database configuration (local or remote)
Intended Hardware to be used - if there is any (Model, Number of CPU’s, CPU MHz, RAM, Disks)
Version of Control-M/Server that is Intended to be Installed

Control-M SQL Queries

You may watch the video here 
https://www.youtube.com/watch?v=qrC-JhJqfRg

These queries can also be automated using Control-M database. Run these queries against enterprise manager database. I have tested it on postgres. You may need to change syntax a bit as per database.

1--> To check number of jobs loaded daily

select net_date, jobs from net_report;

2--> To check number of jobs executed today 

SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status
from runinfo_history
where start_time > Current_Date

3--> To check number of jobs executed between a time period

SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status
from runinfo_history
where start_time  between '2016-09-01 00:00:00' and '2016-09-10 23:59:59'

4--> To check number of failed jobs today


SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status
from runinfo_history
where start_time > Current_Date
and ended_status=32

5--> To check number of failed jobs between a time period

SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status
from runinfo_history
where start_time  between '2016-09-01 00:00:00' and '2016-09-10 23:59:59'
and ended_status=32


6--> To check number of jobs which completed successfully today

SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status
from runinfo_history
where start_time > Current_Date
and ended_status=16

7--> To check number of jobs which completed successfully between a time period

SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status
from runinfo_history
where start_time  between '2016-09-01 00:00:00' and '2016-09-10 23:59:59'
and ended_status=16

8--> To check number of jobs which executed on a particular server in a time period

SELECT job_mem_name , sched_table as folder, application, start_time, end_time,
case when ended_status=16 then 'Ended OK' else 'Not Ok' end as Status , node_id as Server
from runinfo_history
where start_time  between '2016-09-01 00:00:00' and '2016-09-10 23:59:59'
and ended_status=16 and node_id='<Server IP or Hostname>'


9--> To check count of jobs which ended ok and not ok today

select 'OK' Status, sum( case when ended_status=16 then 1 else 0 end) as Count from runinfo_history where ended_status=16
and start_time > current_date
group by ended_status

union all

select 'Not OK' Status, sum( case when ended_status=32 then 1 else 0 end) as Count from runinfo_history where ended_status=32
and start_time> current_date
group by ended_status


10--> To check count of success and failed jobs for a particular application folder

select * from
(
select 'OK' Status, sum( case when ended_status=16 then 1 else 0 end) as Count from runinfo_history where ended_status=16
and start_time Between '2016-09-01 00:00:00' and '2016-09-30 23:59:59' and application like '%a%'
group by ended_status

union all

select 'Not OK' Status, sum( case when ended_status=32 then 1 else 0 end) as Count from runinfo_history where ended_status=32
and start_time Between '2016-09-01 00:00:00' and '2016-09-30 23:59:59' and application like '%a%'
group by ended_status
) Table1


11--> To check count of success and failed jobs , application wise

select * from
(
select application, 'OK' Status, sum( case when ended_status=16 then 1 else 0 end) as Count from runinfo_history where ended_status=16
and start_time Between '2016-09-01 00:00:00' and '2016-09-30 23:59:59'
group by application, ended_status

union all

select application, 'Not OK' Status, sum( case when ended_status=32 then 1 else 0 end) as Count from runinfo_history where ended_status=32
and start_time Between '2016-09-01 00:00:00' and '2016-09-30 23:59:59'
group by application, ended_status
) Table1
order by 1

Control-M Version 9: What's new in Control-M Version 9

Control-M Version 9 Features

High Availability

Control-M high availability solution enables you to maximize your production environment uptime and prevent data loss in the event of hardware maintenance or failure.
In the case of hardware failure or if all Control-M processes are down unexpectedly, you can now perform either an automatic or manual failover for both Control-M/EM and Control-M/Server. After the problem is fixed, you can fallback to the primary host. This functionally applies to customers using an Oracle or MSSQL database for Control-M.
In the event of a hardware failure, when using Control-M/Server with PostgreSQL database, you can perform a manual failover to a secondary Control-M/Server, and later, when the problem is fixed, you can fallback to the primary Control-M/Server. In addition, this solution provides PostgreSQL database replication for Control-M/Server.


Automated Agent Deployment

Upgrade one or many agents to V9 and future versions including fix packs in just a few simple steps. You can choose between transferring the package to agents for later deployment, or automatically transferring and deploying in a single step.

Automated Client Deployment

The new Client Deployment utility replaces the WebLaunch tool in previous Control-M versions. Use the Client Deployment utility to easily distribute full client installation packages, fix packs and patches to users. You can set policies to force users to apply update packages by a specific date.


Promotion between environments

Promotion between environments is a new capability of the Control-M Workload Change Manager that enables you to move job definitions between environments. For example, from a development environment to a production environment. During the promotion process, job definitions are modified and validated according the Site Standards of the target environment.

Control-M configuration Manager (CCM) enhancements

The Control-M configuration Manager user interface has been updated to improve user experience. In addition, the following Control-M components are now managed via the tool: Control-M Database, Naming Server (NS), Configuration Management Server (CMS) as well as Control-M Workload Archiving.


Usage alerts

Keep an eye on your task usage with Usage Alerts, available from within the CCM. Define task usage alert thresholds across all of your EMs. Exceeding a threshold will send an email alert.

Security enhancements

Introducing the SSL certificate deployment capability that extends the packaging and deployment of SSL security certificates with the option to specify an external Certificate Authority.
Improved security algorithmSSL support for:
• AES suite ciphers (128 bit and 256 bit)
• SHA-2 hashing (on Enterprise Manager Client-Server communication)
• RSA keys with length 2048, 3072 and 4096
• Control-M Enterprise Manager password hashed using SHA-512


NFS support for Control-M/Server and Control-M Agent installation

You can now install Control-M/Server and Control-M/Agents on NFS.


IPv6 communication

IPv6 support, including support for mixed IPv4 and IPv6 environments, is now available for the following components:
• Control-M/Agent
• Control-M/Server
• Control-M Mainframe
• Control-M/Enterprise Management – Server, Clients
Note: Control-M Agentless is not supported at this configuration.


Sharing information between jobs

Ability to share information between jobs by extracting data from a job output and passing it to other jobs as a variable. Additional granularity of the variables is introduced as well, allowing to define variables in groups called “named pools” or as Smart folder variables that are unique per folder execution, in addition to global and local variables.


Runtime estimations

With Control-M Batch Impact Manager you can now see estimated runtimes of all the jobs in a regular viewpoint without them being defined as a part of a Batch Impact Manager service.


Multi-days job flows monitoring

Improved monitoring and troubleshooting for the job flows that span on multiple days. You can now configure the active environment retention period for a SMART folder so the SMART folder itself and its jobs are kept in the active environment until the end of the period.


Dynamic job ordering on a remote Control-M Server

Ability to order jobs on remote Control-M Servers by setting an action in a job to trigger another job on a different Control-M Server and also pass variables as a part of this order operation.


Control-M Event Management Rules in Workload Automation client

z/OS customers can now view, edit, and manage CMEM rules via the Control-M Workload Automation client.


What If Host simulation

New capability for Control-M Batch Impact Manager and Forecast to simulate the influence of the unavailability of a specific Control-M Agent or a group of Control-M Agents on your job flows, and prepare for planned shutdowns and maintenance windows.


Global calendars

You can now save time and reduce the efforts invested in calendar management by creating a single calendar in the Control-M/Enterprise Manager that will be distributed to all the Control-M/Servers.


Bookmarks

You can bookmark the jobs in a workspace or a viewpoint to analyze flows and easily locate your jobs. Bookmarks are supported in the Planning, Monitoring, Forecast, and History domains.


Export flow diagram to a file

You can now export your flow diagram to a Microsoft Visio-compatible EMF format from any domain. Just click the “Save as” function in the application “File” menu.


Enhanced search when loading jobs to a workspace

Now you can view more fine-tuned results when loading jobs into a workspace in the Planning domain by filtering folders and jobs with a larger variety of fields.

Automating Control-M database hot backups

As an administrator you may want to automate your database backup jobs in order to avoid manual effort. Here are the steps to do that:

Prior to taking hot backup make sure archival is on. You may create a Control-M OS type job to automate it to take hot backup of Control-M database periodically. Use DBUHotBackup utility for that. Path for this utility is as given below.

<Control-M server home directory>\BMC Software\Control-M Server\ctm_server\exe\DBUtils\DBUHotBackup

e.g. If installation is done in C:\Program Files then path will be as below:

C:\Program Files\BMC Software\Control-M Server\ctm_server\exe\DBUtils\DBUHotBackup


Syntax:

DBUHotBackup -BACKUP_DIRECTORY <full path of backup directory> -ADMINISTRATOR_PASSWORD <database administrator password>

e.g.

DBUHotBackup -BACKUP_DIRECTORY e:\ctmbackups\ -ADMINISTRATOR_PASSWORD Ctm123

One prerequisite is that your backup directory folder should be empty. So you need to make sure you move the files after taking backups to another directory. You may schedule this job to run daily at non peak hours. 

Control-M Hot backup

Control-M archiving requires to be turned on in order to be able to take hot backup of database. Hot database backups jobs can also be created and scheduled by Control-M administrator so that backup is taken periodically and without any manual intervention.

Below is the steps to turn on Archiving ON and take Hot backup.

In order to take hot backup, archiving must be on. So first steps is to turn archiving on. You need to give path of a directory where archiving files will be getting created periodically. So Prior to turn Archiving on make sure you have created folder where you want your archiving files to be stored.

Steps to turn archiving on:

a.       Run command prompt as an administrator

b.      Run “ctm_menu” from command prompt. Select “database menu” from list of available options

a.       Select “management”

Select “Set Database Archive Mode” . In Mode give “ON” in Archive Directory option give “<path of your directory where you want to keep archive files>”.

You need to restart database for changes to take effect. So select “stop database” and after that select “start database” so that database restarts.

Now select "q" to quit and go back to previous options. Select "maintenance" and you will be presented with the below options:

Select "Hot Database Backup". It will ask for the Backup directory and database administrator password. Provide those details so that backup is successfully initiated. Hot backup generally does not take more than 15 minutes for medium sized environment.

Note: Control-M hot backup does not take require any downtime. 

Hope it helps. Please comment if you need any help.

Control-M Advance scheduling: Implementing maybe conditions

There are times when we want our job dependencies to consider only the jobs which are loaded and available in Monitoring(AJF).
E.g. We may have daily jobs which are dependent on weekly, or monthly jobs when weekly or monthly jobs are running, on other days daily jobs should run without checking conditions of weekly or monthly jobs.

Fortunately Control-M provides a way to set up such kind of jobs using Maybe conditions:

Steps to implement this:
1. Let's suppose we have Job1, Job2, and Job3
2. Job1 is weekly job and runs on each saturday.
3. Job2 is a monthly job and runs first of each month.
4. Job3 is a daily job which runs everyday.
5. Job 3 waits for weekly and monthly jobs to finish when weekly (Job1) and monthly (Job2) jobs are in AJF.


Such scenario's are implemented using Control-M "ctmldnrs" Utility.

This is how it works:
1. Ctmldnrs scans each jobs's conditions (whenever this utility is run) and finds out pre-req conditions which are not going to be loaded because those jobs are not scheduled to run that day. It creates a default condition file under <control-m server home directory>/ctm_server/ctmldnrs.dat

2. Load the conditions which are not going to be loaded because of jobs not being in ajf, using "ctmldnrs -load"



Creating Job1, Job2, Job3

You can create two jobs and automate it to run every day. Both the jobs will be command jobs and will be running on Control-M server machine. Commands for jobs are.

Condition_Scan_Job
1. ctmldnrs -CALCCOND -ADDMODE NO

Condition_Load_Job
2. ctmldnrs -LOAD "#-*"

Note: Make Condition_Load_Job dependent on Condition_Scan_Job. Schedule Condition_Scan_Job to run immediately after NDP everyday.

So here all the conditions which are set with #- prefix will be loaded if are not available because of job being not available in monitoring(ajf) because of it's scheduling criteria not met.