Model Searching#

In this example, we will use the permeability data in 1A, “ADME Properties Evaluation in Drug Discovery: Prediction of Caco-2 Cell Permeability Using a Combination of NSGA-II and Boosting”, and instead of defining a model ourselves, we will search over a list of top model architectures and find the best one.

# We'll first load in the data, same as in 1A

import requests
import os

if not os.path.exists("caco2_data.xlsx"):
    r = requests.get("")
    open("caco2_data.xlsx" , 'wb').write(r.content)

# Reading the data into a dataframe
# Subsetting the data into molecule, split, and property
# Converting property values to floats
# Creating splits

import pandas as pd
import numpy as np

df = pd.read_excel("caco2_data.xlsx")

df["split"] = df["Dataset"].replace({"Tr": "train", "Te": "test"})
df = df[["smi", "split", "logPapp"]].dropna()

def isfloat(num):
        return True
    except ValueError:
        return False
df = df[df["logPapp"].apply(isfloat)]

df["logPapp"] = df["logPapp"].astype('float')

# Now we use the dataframe to create a BaseDataset object.
# We will generate it from the pd.DataFrame object.
# We have defined our own split column, which will be used by the dataset object.

import olorenchemengine as oce

dataset = oce.BaseDataset(data = df.to_csv(), structure_col="smi", property_col="logPapp")
# We'll now get our list of top model architectures.
# Each of these models has certain situations where it outperforms the others,
# so we test all of them to see which model is best for this specific situation.
models = oce.TOP_MODELS_ADMET()

# We'll also create a ModelManager object to keep track of our experiments
mm = oce.ModelManager(dataset, metrics = ["Root Mean Squared Error"], file_path="mm_1B_results.oce")
# This will now use our model manager to test our top models
# and will take around 1-4 hours to run in total for this dataset,
# though it will take more or less time depending on the machine.
# Get the list of models and sort by their RMSE performance metrics
# We see that the best model now outperforms the published models.

mm.get_model_database().sort_values(by="Root Mean Squared Error")
Model Name Model Parameters Fitting Time Root Mean Squared Error
0 ZWK_XGBoostModel 8t4Lbm1C {'BC_class_name': 'ZWK_XGBoostModel', 'args': ... 583.755889 0.306506
4 RFStacker ZObB1n2V {'BC_class_name': 'RFStacker', 'args': [[{'BC_... 1370.884091 0.326439
2 BaseBoosting sSOI0-2O {'BC_class_name': 'BaseBoosting', 'args': [[{'... 80.049373 0.332818
6 RFStacker Dg3XrFow {'BC_class_name': 'RFStacker', 'args': [[{'BC_... 1077.793208 0.344461
9 ZWK_XGBoostModel u3zq9AAV {'BC_class_name': 'ZWK_XGBoostModel', 'args': ... 583.068085 0.350327
1 BaseBoosting GDDXgNxr {'BC_class_name': 'BaseBoosting', 'args': [[{'... 98.140028 0.354108
13 RFStacker J-KhwR5S {'BC_class_name': 'RFStacker', 'args': [[{'BC_... 2523.654553 0.378332
10 BaseBoosting 1zpI0dIb {'BC_class_name': 'BaseBoosting', 'args': [[{'... 31.493627 0.390516
11 BaseBoosting ADkCCrwJ {'BC_class_name': 'BaseBoosting', 'args': [[{'... 4.215762 0.402029
15 RFStacker kHyqmLCI {'BC_class_name': 'RFStacker', 'args': [[{'BC_... 4003.144378 0.450928
7 ResampleAdaboost rw2YnX2a {'BC_class_name': 'ResampleAdaboost', 'args': ... 150.905021 0.452858
14 BaseBoosting Q-ko4Uuj {'BC_class_name': 'BaseBoosting', 'args': [[{'... 70.730023 0.460666
16 ResampleAdaboost rw2YnX2a {'BC_class_name': 'ResampleAdaboost', 'args': ... 432.079354 0.474882
12 SPGNN TWy3l_kb {'BC_class_name': 'SPGNN', 'args': [], 'kwargs... 9.815435 0.487068
5 BaseBoosting Px-cadEt {'BC_class_name': 'BaseBoosting', 'args': [[{'... 54.437475 0.492785
8 KNN 20fz7vhA {'BC_class_name': 'KNN', 'args': [{'BC_class_n... 1.745354 0.526132
17 KNN 20fz7vhA {'BC_class_name': 'KNN', 'args': [{'BC_class_n... 0.016418 0.926369
3 SPGNN 8PvbRqPX {'BC_class_name': 'SPGNN', 'args': [], 'kwargs... 10.181282 2.647019