The Air Force is not alone in being a modern organization struggling to effectively process all the data it has at its disposal. The Air Force, however, is one of the very few organizations where rapidly turning data into information, knowledge, decisions, and actions has national security implications. Due to a number of technological improvements, perhaps most notably the proliferation of data streaming IoT sensors/devices and additional intelligence collections channels producing higher and higher resolution, there is no envisioned end to the deluge of complex data. The Air Force requires a means to turn the complexity of their data-rich environment into a strategic advantage. The convergence of Cloud Computing, Artificial Intelligence (AI), and Data Analytics offers such a means. Leveraging these means in a creative manner enables something Cougaar Software, Inc. (CSI) refers to as AI-Curated Knowledge. More significantly, AI-Curated Knowledge offers the Air Force the ability to conquer complexity and attain a decisive data utilization advantage.
Fundamentally, AI-Curated Knowledge leverages the power of agents, operating as a distributed society, with each agent responsible for and an expert in processing a specific class, or association, of data. Individual agents employ a variety of AI and data analytics techniques, including machine learning, pattern analysis, case-based reasoning, rules, knowledge discovery, semantic reasoning, and inference. Each agent serves as a curator or librarian for its data – all data coming into the data repository is reviewed, validated, verified, tagged with meta-data and processed by the librarian before being “filed.” Any request for a specific repository’s information (from another agent or from a human collaborator) is routed to its curator, who will verify access and authorization and processes the request to pull the right information and package it into the applicable form to fulfill the request. Multiple copies of an agent may be used for high demand data types. Multiple repositories may be maintained – partitioned or mirrored – to support the locality of data access and performance/scalability requirements.
The repository itself is a secure, scalable hybrid knowledge graph, combining conventional relation data stores with graph data stores and semantic knowledge stores to create a linked representation that supports efficient numeric, textual, semantic, and pattern analysis. The knowledge graph maintains the many node types and relationship types associated with its information, as well as meta-information generated by supporting agents describing the aggregations, statistics, and cluster populations of information – organized in temporal, geographic, organizational, and other dimensions. The payloads of the nodes are references to the multi-faceted data objects with semantic tagging, property markers, and meta-data for context, pedigree, and providence.
Repositories, these hybrid knowledge graphs, are at the core of AI-Curated Knowledge. Dozens of agents serve as the librarians managing, marking, cleaning, processing, and updating this core. Once developed, agents and the core are most powerfully situated in the cloud, allowing the elastic expansion of computational power, storage, and bandwidth to meet the needs of the using community, when and where needed. The community accesses the curated knowledge store via micro-services, small, composable interfaces that provide appropriately configured access with the right parameters, and returning in the right form. Micro-services comply with industry standards and support both SOAP and RESTful interfaces, with XML and JSON respectively. Each micro-service is invoked with a security token, or delegate, to ensure every transaction request is authenticated and authorized in the contextual Role-Based Access Control (RBAC) scheme.
The use of micro-services as access points allows any authorized system, device, or user to post data or information using easy to develop and open standards. Further, any authorized system, device, or the user can pull data or information, or invoke a query or service, using the same interfaces. The key, as noted earlier, is that the recipient of the micro-service call is a cognitive agent capable of reasoning over, processing, and coordinating the post or request. In some cases, a post or request may require the agent to coordinate with one or more other agents, performing some advanced collaborative planning, reasoning, or fusion – but appearing to the outside world like a simple micro-service interface.
This is how I propose the Air Force leverage data to its strategic advantage, establish a Data Architecture for long-term care and maintenance of data, information, and knowledge, and allow secure, controlled access to information at every echelon. With its vast troves of data and technological head start, the Air Force has the opportunity to turn the challenge of its complex data environment into an indomitable strength. AI-Curated Knowledge, I believe, offers the best, most cost-effective and future-proof means to achieve that goal.