Security Force Monitor follows a four phase process when researching the security forces of a country.
Phase 1: Scope out sources
Security Force Monitor collects data about the persons and organizations that comprise security forces, along with allegations of human rights abuses committed by security forces. This data is carefully collected from a variety of sources, generally online. These include:
- Laws of the country;
- Official government media;
- Press releases from the relevant ministries of the country (Information, Defense, Interior, and others);
- Security force newsletters;
- Social media pages for security services or government agencies;
- Statistics and data agencies;
- Local government websites;
- Human rights commissions;
- United States Government reporting;
- United Nations reporting;
- Local news reportage;
- Civil society and human rights reporting;
- Academic research; and,
- Other country specific sources.
The Monitor also identifies non-digital resources such as monographs, scholarly literature, biographies and other materials about security services. The existence and availability of these type of sources vary widely from country to country.
State administrative structures and geography
Additionally, Monitor researchers familiarize themselves with the country’s governance and administrative structures, gaining understanding about the levels of government (for example: local, regional, state, national) and their connection to different security forces. In examining this we also flag where there might be major changes in the structure of government, such as those that may accompany a constitutional referendum or a peace process. These affect how the Monitor will represent data on security forces over time.
Examining the administrative geography of a country provides important context for the structure and operations of the security forces. This part of the scoping process also gives us insight into how much of a country’s administrative geography is represented in online gazetteers (like OpenStreetMap or GeoNames) that the Monitor uses in its analysis.
Potential high value datasets
During this first phase, researchers also identify sources that could be turned into large, high value datasets for the Monitor. These are sources that contain a sufficiently large amount or complex type of data that technical help is necessary to extract it in a timely way; doing the work “by hand” is possible, but would be slow and error prone. These high value data extraction tasks use techniques such as web-scraping, scripted parsing and geospatial analysis. A good examples of this sort of opportunity is:
- Over 10 years of Internet Archive snapshots of various versions of the official Ejército Mexicano webpage (and child pages), which outline the top level structures of the Mexican army and its commanding officers.
Phase 2: Write a Country Guide
The sources gathered during our scoping phase are used to write a Country Guide to assist with further research.
The Guide begins as a general overview of the structures of security services of a country. It gives researchers a general framework to help organize and prioritise research, by giving an estimate of the scale of the work: number of installations, organizations, persons that need to be researchers in detail.
For example, from our scoping research on Nigeria we know that by law every state and the federal territory has a Police Command which means since 1996 (when the last states were created) there are 37 Police Commands. Statistics from the Nigeria Police Force show that between 2007 and 2012 most Police Commands were in charge of 2 to 3 Police Area Commands (though some heavily populated states have more, with Lagos having the most at 8). Further these statistics show that on average each state and the federal capital territory had 30 Police Divisions, which by law are generally under the command of a Police Area Command.
Further, a list of keywords and sources is created for researchers as well - these serve as leads for researchers to follow.
Phase 3: Conduct detailed research
Researchers use the initial keywords and sources to begin a “deep dive” into the security forces.
Anything relating to the types of information the Monitor collects is entered into the Monitor’s database. The Data Model section of this Research Handbook gives detailed guidance to Monitor researchers about the types of data to take from sources and how it should be entered.
Additionally, data from this research is used to update the Country Guide to provide greater granular detail on the security forces of a country or to update sections as needed. New sources and keywords discovered during research are added to the existing guide as well.
As the Monitor builds its database on the security forces of a country, new information is cross-referenced to corroborate data, discover gaps, or identify mistakes. There are three key questions Monitor researchers ask themselves as they work:
- "Does this make sense given our consensus knowledge?"
- "Is it possible this source is correct and our consensus knowledge is wrong?"
- "Do I have enough information to accept or reject what this source says?"
The Country Guide acts as a framework for researchers to understand the security forces and to help a researcher answer these questions. Data entered into the Monitor’s databases are progressively updated as new information is found. Throughout, researchers take a number of quality assurance steps to validate the data against the standards set out in the Data Model section.
Phase 4: Publish data
Researchers will publish data online when confident that the main branches and overall structure of a country’s security forces are adequately covered. In April 2017, the Monitor released in draft for comment and feedback datasets on Mexico and Nigeria. The data was published on a prototype data visualization platform. Based on user feedback, Security Force Monitor released WhoWasInCommand.com, a substantially updated version of is web plaform.