Research methodology

Security Force Monitor follows a four phase process when researching the security forces of a country.

Phase 1: Scope out sources

General sources

Security Force Monitor collects data about the Persons and Units that comprise security forces, along with allegations of human rights abuses made against security forces. This data is carefully collected from a variety of sources, generally online. These include:

  • Laws of the country;
  • Official government media;
  • Press releases from the relevant ministries of the country (Information, Defense, Interior, and others);
  • Security force newsletters;
  • Social media pages for security services or government agencies;
  • Statistics and data agencies;
  • Local government websites;
  • Human rights commissions;
  • Third country government publications and other documents;
  • United Nations publications and other documents;
  • Local news reportage;
  • Civil society and human rights reporting;
  • Academic research; and,
  • Other country specific sources.

The Monitor also identifies non-digital resources such as monographs, scholarly literature, biographies and other materials about security services. The existence and availability of these type of sources vary widely from country to country.

State administrative structures and geography

Security Force Monitor researchers familiarize themselves with the country’s governance and administrative structures, gaining understanding about the levels of government (for example: local, regional, state, national) and their connection to different security forces. In examining this we also flag where there might be major changes in the structure of government, such as those that may accompany a constitutional referendum or a peace process. These affect how the Monitor will represent data on security forces over time.

Examining the administrative geography of a country provides important context for the structure and operations of the security forces. This part of the scoping process also gives us insight into how much of a country’s administrative geography is represented in online gazetteers (like OpenStreetMap or GeoNames) that the Monitor uses in its analysis.

Potential high value datasets

During this first phase, researchers also identify sources that could be turned into large, high value datasets for the Monitor. These are sources that contain a sufficiently large amount or complex type of data that technical help is necessary to extract it in a timely way; doing the work “by hand” is possible, but would be slow and error prone. These high value data extraction tasks use techniques such as web-scraping, scripted parsing and geospatial analysis. An example of these types of sources are the Internet Archive snapshots the official webpage of the Mexican Army and Air Force (and child pages) going back more than a decade which outline the top level structures of the Mexican army and its commanding officers.

Phase 2: Write a Country Guide

The sources gathered during our scoping phase are used to write a Country Guide to assist with further research.

The Guide begins as a general overview of the structures of security services of a country. It gives researchers a general framework to help organize and prioritize research, by giving an estimate of the scale of the work: number of installations, units, persons that need to be researchers in detail.

For example, the Monitor’s scoping research on Nigeria revealed that by law every state and the federal territory has a Police Command. By extension this means since 1996 (when the last states of Nigeria were created) there have been 37 Police Commands. Statistics from the Nigeria Police Force show that between 2007 and 2012 most Police Commands were in charge of 2 to 3 Police Area Commands (though some heavily populated states have more, with Lagos having the most at 8). These statistics also show that on average each state and the federal capital territory had 30 Police Divisions, which by law are generally under the command of a Police Area Command. Thus, from a few initial sources a Monitor researcher can have a general sense of the structure of the police and a useful metric to compare the Monitor’s dataset against.

Further, a list of keywords and sources is created for researchers as well - these serve as leads for researchers to follow.

This Country Guide is updated as new details on the security forces are discovered through the course of the Monitor’s work.

Phase 3: Conduct detailed research

Researchers use the initial keywords and sources to begin a “deep dive” into the security forces.

Anything relating to the types of information the Monitor collects is entered into the Monitor’s database. The Data Model section of this Research Handbook gives detailed guidance to Monitor researchers about the types of data to take from sources and how it should be entered.

Additionally, data from this research is used to update the Country Guide to provide greater granular detail on the security forces of a country or to update sections as needed. New sources and keywords discovered during research are added to the existing guide as well.

As the Monitor builds its database on the security forces of a country, new information is cross-referenced to further evidence data, discover gaps, or identify mistakes. There are three key questions Monitor researchers ask themselves whenever they review information in a source:

  • “Does this make sense given what is detailed about the security forces from other sources?”
  • “Is it possible this source is correct and our other sources are incorrect?”
  • “Do I have enough information to accept or reject what this source says?”

The Country Guide acts as a framework for researchers to understand the security forces and to help a researcher answer these questions. Data entered into the Monitor’s databases are progressively updated as new information is found. Throughout, researchers take a number of quality assurance steps to validate the data against the standards set out in the Data Model section

Phase 4: Publish data

Researchers will publish data online when confident that the main branches and overall structure of a country’s security forces are adequately covered. Data is published online on the Monitor’s platform: