Richard Wen
[email protected]
The nbc4va package implements the Naive Bayes Classifier (NBC) algorithm for verbal autopsy data based on code and methods provided by Miasnikof et al (2015).
This package is intended to be used for experimenting with the NBC algorithm to predict causes of death using verbal autopsy data.
This package was developed at the Centre for Global Health Research (CGHR) in Toronto, Ontario, Canada. The original NBC algorithm code was developed by Pierre Miaskinof and Vasily Giannakeas. The original performance metrics code was provided by Dr. Mireille Gomes whom also offered guidance in metrics implementation and user testing. Special thanks to Richard Zehang Li for providing a standard structure for the package and Patrycja Kolpak for user testing of the GUI.
Run the nbc4va Graphical User Interface (GUI) and follow the instructions in your browser:
Exit the GUI by closing the browser window and pressing
Esc in a R console to stop the GUI process.
Before using the nbc4va package, ensure that the training and testing data inputs are formatted correctly and that the terms used in this package are understood:
The nbc4va package contains help sections with code samples and references for usage:
By using the help() function (or ?
shortform) in an R console, you can access details about particular
functions and methods in the nbc4va package:
library(nbc4va) # load the nbc4va package
# View this help page as a vignette
browseVignettes("nbc4va")
# Access details about certain functions
help("nbc4va") # access the nbc4va package docs
help("nbc4vaGUI") # access GUI details
help("nbc4vaIO") # access file in and out details
help("nbc") # access the nbc algorithm function
help("summary.nbc") # access the summary function
help("plot.nbc") # access the results plot function
# Access details about example data
help("nbc4vaData")
help("nbc4vaDataRaw")
# Alternative short forms
?nbc4va
?nbc4vaGUI
?nbc4vaIO
?nbc
?nbc4vaData
?nbc4vaDataRaw
?summary.nbc
?plot.nbcFor help with bugs, issues, enhancements, and related inquiries, please submit a new issue at:
To view citation information for the nbc4va package, use the code below in an R console:
This section provides details on the basic usage of the nbc4va package which includes bringing up the Graphic User Interface and running the Naive Bayes Classifier algorithm using file input and output.
The simplest way to use the package is to open the Graphical User
Interface (GUI) in your default web browser with
nbc4vaGUI().
Once the GUI is loaded, follow the instructions to fit a NBC model to
your training.csv and to evaluate its performance with your
testing.csv data.
If nbc4vaGUI() is called sucessfully, the GUI shown in
the image below should be available in your web browser.
Run the following code using nbc4vaGUI() in a R console
to open the GUI in your web browser:
Close the GUI by pressing escape while you are in the R console.
See the Methods section for definitions of performance metrics and terms in the model results.
The nbc4vaIO() function can be called to fit a NBC model
and save its results using the paths to your training and
testing files in Comma Separated Values (CSV) format.
The saved results will in a selected directory with four CSV files detailing the performance of the model:
.._pred.csv: a table of predictions, where the columns
Prediction1..PredictionN are the cause of death predictions with
Prediction1 being the most probable cause.._prob.csv: a table of probabilities, where each
column is a cause of death and each cell is the probability of a case
being that cause.._causes.csv: a table of metrics for each cause.._metrics.csv: a table of summary metrics for the
modeltesting
fileThe image below shows the input files on the left and the saved
results on the right using the nbc4vaIO() function (with
the fileHeader argument set to "nbc4va").
Run the following code using nbc4vaIO() in a R console
to produce NBC model performance results with the training
and testing files.
library(nbc4va)
# Find paths to your "trainFile" and "testFile"
trainFile <- file.choose() # select train file first
testFile <- file.choose() # followed by test file after
# Run NBC model
# Dump results to same directory as "testFile"
# Set "known"" to indicate whether testing causes are known
nbc4vaIO(trainFile, testFile, known=TRUE)This section provides details on the advanced usage of the nbc4va package which includes training a NBC model, evaluating NBC model performance, and plotting the top predicted causes from the NBC model.
The documentation written here is intended for users of R that understand the different data structures of R such as:
It is also required to understand the basic data types:
Run the following code using nbc() in a R console to
train a NBC model:
library(nbc4va)
# Create training and testing dataframes
data(nbc4vaData) # example data
train <- nbc4vaData[1:50, ]
test <- nbc4vaData[51:100, ]
# Train a nbc model
# The "results" variable is a nbc list-like object with elements accessible by $
# Set "known" to indicate whether or not testing causes are known in "test"
results <- nbc(train, test, known=TRUE)
# Obtain the probabilities and predictions
prob <- results$prob.causes # vector of probabilities for each test case
pred <- results$pred.causes # vector of top predictions for each test case
# View the "prob" and "pred", the names are the case ids
head(prob)
head(pred)See the Methods section for the NBC algorithm details.
For complete function specifications and usage of nbc(),
use the code below in an R console:
Run the following code using summary.nbc() in a R
console to evaluate a NBC model:
library(nbc4va)
# Create training and testing dataframes
data(nbc4vaData)
train <- nbc4vaData[1:50, ]
test <- nbc4vaData[51:100, ]
# Train a nbc model
results <- nbc(train, test, known=TRUE)
# Automatically calculate metrics with summary
# The "brief" variable is a nbc_summary list-like object
# The "brief" variable is "results", but with additional metrics
brief <- summary(results)
# Obtain the calculated metrics
metrics <- brief$metrics.all # vector of overall metrics
causeMetrics <- brief$metrics.causes # dataframe of metrics by cause
# Access the calculatd metrics
metrics[["CSMFaccuracy"]]
metrics[["Sensitivity"]]
View(causeMetrics)See the Methods section for definitions of performance metrics and terms in the output.
For complete method specifications and usage of
summary.nbc(), use the code below in a R console:
Run the following code using plot.nbc() in a R console
to produce a bar plot of the top predicted causes:
library(nbc4va)
# Create training and testing data
data(nbc4vaData)
train <- nbc4vaData[1:50, ]
test <- nbc4vaData[51:100, ]
# Train a nbc model and plot the top 5 causes if possible
results <- nbc(train, test, known=TRUE)
plot(results, top=5)
plot(results, top=5, footnote=FALSE) # remove footnoteThe image below shows a plot of the top causes of death by predicted
CSMFs using plot.nbc() on a NBC model trained using the
example data nbc4vaData included in the package.
See the Methods section for definition of CSMF and related metrics in the footnote of the plot.
For complete method specifications and usage of
plot.nbc(), use the code below in a R console:
This section provides an organized list of the available functions in the nbc4va package and a brief description of what they are used for.
nbc() model
results with informative metricsnbc() modelnbc()nbc()nbc() modelThis documentation page provides details on the training and testing data formats to be used as inputs in the nbc4va package.
The training data (consisting of cases, causes of death for each case, and symptoms) is used as input for the Naive Bayes Classifier (NBC) algorithm to learn the probabilities for each cause of death to produce a NBC model.
This model can be evaluated for its performance by predicting on the testing data cases, where the predicted causes of death are compared to the causes of death in the testing data.
The process of learning the probabilities to produce the NBC model is known as training, and the process of evaluating the predictive performance of the trained model is known as testing.
Key points:
The format of the training and testing data is structured as a table, where each column holds a variable and each row holds a death case.
The following format must be met in order to be used with the nbc4va package:
Cause can be omittedThe image below shows an example of the training data.
The image below shows an example of the corresponding testing data.
The image below shows an example of the corresponding testing data without any causes.
Given a symptom column containing the values of each case (1, 0, 0, 1, 99, 99):
The imputation is applied as follows:
The symptom imputation method preserves the approximate distribution of the known values in an attempt to avoid dropping entire cases or symptoms.
Run the following code using nbc4vaData() in the R
console to view the example data included in the nbc4va package:
This section provides details on the implementation of the Naive Bayes Classifier algorithm, definition of uncommon terms, and calculation of performance metrics.
The Naive Bayes Classifier (NBC) is a machine learning algorithm that uses training data containing cases of deaths to learn probabilities for known causes of death based on given symptoms. This produces a model that can use the learned probabilities to predict the cause of death for cases in unseen testing data with same symptoms.
The nbc4va package implements the NBC algorithm for verbal autopsy data using code and methods built on Miasnikof et al (2015).
Symptom: Refers to the features or independent variables with binary values of 1 for presence and 0 for absence of a death related condition.
Cause: Refers to the target or dependent variable containing discrete values of the causes of death.
Case: Refers to an individual death containing an identifier, a cause of death (if known), and several symptoms.
Training Data: Refers to a dataset of cases that the NBC algorithm learns probabilities from.
Testing Data: Refers to a dataset of cases used to
evaluate the performance of a NBC model; these cases must have the same
symptoms as the Training Data, but with different
cases.
True Positives: The number of cases, given a cause, where the predicted cause is equal to the actual observed cause (Fawcett, 2005).
True Negatives: The number of cases, given a cause, where the predicted is not the cause and the actual observed is also not the cause (Fawcett, 2005).
False Positives: The number of cases, given a cause, where the predicted is the cause and the actual observed is not the cause (Fawcett, 2005).
False Negatives: The number of cases, given a cause, where the predicted is not the cause and the actual observed is the cause (Fawcett, 2005).
CSMF: The fraction of deaths (predicted or observed) for a particular cause.
The following metrics measure the performance of a model by comparing its predicted causes individually to the matching true/observed causes.
Sensitivity: proportion of correctly identified positives (Powers, 2011).
$$ Sensitivity = \frac{TP}{TP+FN} $$
where:
PCCC: partial chance corrected concordance (Murray et al 2011).
$$ PCCC(k) = \frac{C-\frac{k}{N}}{1-\frac{k}{N}} $$
where:
The following metrics measure the performance of a model by comparing its distribution of cause predictions to a distribution of true/observed causes for similar cases.
CSMFmaxError: cause specific mortality fraction maximum error (Murray et al 2011).
CSMFMaximumError = 2(1 − Min(CSMFjtrue)
where:
CSMFaccuracy: cause specific mortality fraction accuracy (Murray et al 2011).
$$ CSMFAccuracy = 1-\frac{\sum_{j=1}^{k} |CSMF_{j}^{true} - CSMF_{j}^{pred}|}{CSMF Maximum Error} $$
where:
This section provides information for developers on the structure of the code, as well as any package and function dependencies.
nbc()
function argumentssummary.nbc() function argumentsinternalGetCSMFMaxError()internalGetCSMFMaxError(),
internalGetCSMFAcc()internalRoundFixedSum()internalCheckNBC(),
internalNBC()nbc() to calculate informative metrics \itemize{
internalCheckNBCSummary(),
internalGetCauseMetrics(),
internalGetMetrics(),
internalGetCSMFMaxError(),
internalGetCSMFAcc()summary.nbc() to print a message of the top causes
summary.nbc(),
nbc()nbc()
to plot the top causes
summary.nbc(),
nbc()summary.nbc(),
nbc()nbc4vaIO()nbc()
to get the predicted csmfs
nbc(),
summary.nbc()nbc()
to get the top predictions
nbc()