C++ 决策树实现问题:在代码中思考

Posted

技术标签:

【中文标题】C++ 决策树实现问题:在代码中思考【英文标题】:C++ Decision Tree Implementation Question: Think In Code 【发布时间】:2011-08-04 12:07:00 【问题描述】:

我已经编码了几年,但我仍然没有掌握伪编码的窍门,也没有真正用代码思考问题。由于这个问题,我很难弄清楚在创建学习决策树时要做什么。

这里有几个我看过的网站相信我还有很多

Decision Tree Tutorials

DMS Tutorials

还有几本书,例如 Ian Millington 的《AI for Games》,其中包括对决策树中使用的不同学习算法的详细介绍,以及《游戏编程的行为数学》(基本上都是关于决策树和理论的)。我了解决策树的概念以及熵、ID3 以及如何将遗传算法交织在一起并让决策树决定 GA 的节点。 他们提供了很好的洞察力,但不是我真正想要的。

我确实有一些为决策树创建节点的基本代码,而且我相信我知道如何实现实际逻辑,但如果我对程序没有目的或没有熵或学习算法,那它就没有用了涉及。

我要问的是,有人可以帮我弄清楚我需要做什么来创建这个学习决策树。我的节点在自己的类中流过函数来创建树,但是我如何将熵放入其中,如果它有一个类,一个结构,我不知道如何将它放在一起。伪代码和我对所有这些理论和数字的去向的想法。只要我知道我需要编写什么代码,我就可以将代码放在一起。任何指导将不胜感激。

基本上,我该怎么做。

添加学习算法,例如 ID3 和熵。应该如何设置?

一旦我弄清楚了如何处理这一切,我计划将其实现到一个状态机中,该状态机以游戏/模拟格式经历不同的状态。所有这些都已经设置好了,我只是想这可以是独立的,一旦我弄清楚了,我就可以将它移到另一个项目中。

这是我现在拥有的源代码。

提前致谢!

Main.cpp

int main()

    //create the new decision tree object
    DecisionTree* NewTree = new DecisionTree();

    //add root node   the very first 'Question' or decision to be made
    //is monster health greater than player health?
    NewTree->CreateRootNode(1);

    //add nodes depending on decisions
    //2nd decision to be made
    //is monster strength greater than player strength?
    NewTree->AddNode1(1, 2);

    //3rd decision
    //is the monster closer than home base?
    NewTree->AddNode2(1, 3);

    //depending on the weights of all three decisions, will return certain node result
    //results!
    //Run, Attack, 
    NewTree->AddNode1(2, 4);
    NewTree->AddNode2(2, 5);
    NewTree->AddNode1(3, 6);
    NewTree->AddNode2(3, 7);

    //Others: Run to Base ++ Strength, Surrender Monster/Player, 
    //needs to be made recursive, that way when strength++ it affects decisions second time around DT

    //display information after creating all the nodes
    //display the entire tree, i want to make it look like the actual diagram!
    NewTree->Output();

    //ask/answer question decision making process
    NewTree->Query();

    cout << "Decision Made. Press Any Key To Quit." << endl;

    //pause quit, oh wait how did you do that again...look it up and put here

    //release memory!
    delete NewTree;

    //return random value
    //return 1;

决策树.h

//the decision tree class
class DecisionTree

public:
    //functions
    void RemoveNode(TreeNodes* node);
    void DisplayTree(TreeNodes* CurrentNode);
    void Output();
    void Query();
    void QueryTree(TreeNodes* rootNode);
    void AddNode1(int ExistingNodeID, int NewNodeID);
    void AddNode2(int ExistingNodeID, int NewNodeID);
    void CreateRootNode(int NodeID);
    void MakeDecision(TreeNodes* node);

    bool SearchAddNode1(TreeNodes* CurrentNode, int ExistingNodeID, int NewNodeID);
    bool SearchAddNode2(TreeNodes* CurrentNode, int ExistingNodeID, int NewNodeID);

    TreeNodes* m_RootNode;

    DecisionTree();

    virtual ~DecisionTree();
;

Decisions.cpp

int random(int upperLimit);

//for random variables that will effect decisions/node values/weights
int random(int upperLimit)

    int randNum = rand() % upperLimit;
    return randNum;


//constructor
//Step 1!
DecisionTree::DecisionTree()

    //set root node to null on tree creation
    //beginning of tree creation
    m_RootNode = NULL;


//destructor
//Final Step in a sense
DecisionTree::~DecisionTree()

    RemoveNode(m_RootNode);


//Step 2!
void DecisionTree::CreateRootNode(int NodeID)

    //create root node with specific ID
    // In MO, you may want to use thestatic creation of IDs like with entities. depends on how many nodes you plan to have
    //or have instantaneously created nodes/changing nodes
    m_RootNode = new TreeNodes(NodeID);


//Step 5.1!~
void DecisionTree::AddNode1(int ExistingNodeID, int NewNodeID)

    //check to make sure you have a root node. can't add another node without a root node
    if(m_RootNode == NULL)
    
        cout << "ERROR - No Root Node";
        return;
    

    if(SearchAddNode1(m_RootNode, ExistingNodeID, NewNodeID))
    
        cout << "Added Node Type1 With ID " << NewNodeID << " onto Branch Level " << ExistingNodeID << endl;
    
    else
    
        //check
        cout << "Node: " << ExistingNodeID << " Not Found.";
    


//Step 6.1!~ search and add new node to current node
bool DecisionTree::SearchAddNode1(TreeNodes *CurrentNode, int ExistingNodeID, int NewNodeID)

    //if there is a node
    if(CurrentNode->m_NodeID == ExistingNodeID)
    
        //create the node
        if(CurrentNode->NewBranch1 == NULL)
        
            CurrentNode->NewBranch1 = new TreeNodes(NewNodeID);
        
        else 
        
            CurrentNode->NewBranch1 = new TreeNodes(NewNodeID);
        
        return true;
    
    else
    
        //try branch if it exists
        //for a third, add another one of these too!
        if(CurrentNode->NewBranch1 != NULL)
        
            if(SearchAddNode1(CurrentNode->NewBranch1, ExistingNodeID, NewNodeID))
            
                return true;
            
            else
            
                //try second branch if it exists
                if(CurrentNode->NewBranch2 != NULL)
                
                    return(SearchAddNode2(CurrentNode->NewBranch2, ExistingNodeID, NewNodeID));
                
                else
                
                    return false;
                
            
        
        return false;
    


//Step 5.2!~    does same thing as node 1.  if you wanted to have more decisions, 
//create a node 3 which would be the same as this maybe with small differences
void DecisionTree::AddNode2(int ExistingNodeID, int NewNodeID)

    if(m_RootNode == NULL)
    
        cout << "ERROR - No Root Node";
    

    if(SearchAddNode2(m_RootNode, ExistingNodeID, NewNodeID))
    
        cout << "Added Node Type2 With ID " << NewNodeID << " onto Branch Level " << ExistingNodeID << endl;
    
    else
    
        cout << "Node: " << ExistingNodeID << " Not Found.";
    


//Step 6.2!~ search and add new node to current node
//as stated earlier, make one for 3rd node if there was meant to be one
bool DecisionTree::SearchAddNode2(TreeNodes *CurrentNode, int ExistingNodeID, int NewNodeID)

    if(CurrentNode->m_NodeID == ExistingNodeID)
    
        //create the node
        if(CurrentNode->NewBranch2 == NULL)
        
            CurrentNode->NewBranch2 = new TreeNodes(NewNodeID);
        
        else 
        
            CurrentNode->NewBranch2 = new TreeNodes(NewNodeID);
        
        return true;
    
    else
    
        //try branch if it exists
        if(CurrentNode->NewBranch1 != NULL)
        
            if(SearchAddNode2(CurrentNode->NewBranch1, ExistingNodeID, NewNodeID))
            
                return true;
            
            else
            
                //try second branch if it exists
                if(CurrentNode->NewBranch2 != NULL)
                
                    return(SearchAddNode2(CurrentNode->NewBranch2, ExistingNodeID, NewNodeID));
                
                else
                
                    return false;
                
            
        
        return false;
    


//Step 11
void DecisionTree::QueryTree(TreeNodes* CurrentNode)

    if(CurrentNode->NewBranch1 == NULL)
    
        //if both branches are null, tree is at a decision outcome state
        if(CurrentNode->NewBranch2 == NULL)
        
            //output decision 'question'
            ///////////////////////////////////////////////////////////////////////////////////////
        
        else
        
            cout << "Missing Branch 1";
        
        return;
    
    if(CurrentNode->NewBranch2 == NULL)
    
        cout << "Missing Branch 2";
        return;
    

    //otherwise test decisions at current node
    MakeDecision(CurrentNode);


//Step 10
void DecisionTree::Query()

    QueryTree(m_RootNode);


////////////////////////////////////////////////////////////
//debate decisions   create new function for decision logic

// cout << node->stringforquestion;

//Step 12
void DecisionTree::MakeDecision(TreeNodes *node)

    //should I declare variables here or inside of decisions.h
    int PHealth;
    int MHealth;
    int PStrength;
    int MStrength;
    int DistanceFBase;
    int DistanceFMonster;

    ////sets random!
    srand(time(NULL));

    //randomly create the numbers for health, strength and distance for each variable
    PHealth = random(60);
    MHealth = random(60);
    PStrength = random(50);
    MStrength = random(50);
    DistanceFBase = random(75);
    DistanceFMonster = random(75);

    //the decision to be made string example: Player health: Monster Health:  player health is lower/higher
    cout << "Player Health: " << PHealth << endl;
    cout << "Monster Health: " << MHealth << endl;
    cout << "Player Strength: " << PStrength << endl;
    cout << "Monster Strength: " << MStrength << endl;
    cout << "Distance Player is From Base: " << DistanceFBase << endl;
    cout << "Disntace Player is From Monster: " << DistanceFMonster << endl;

    //MH > PH
    //MH < PH
    //PS > MS
    //PS > MS
    //DB > DM
    //DB < DM

    //good place to break off into different decision nodes, not just 'binary'

    //if statement lower/higher query respective branch
    if(PHealth > MHealth)
    
    
    else
    
    
    //re-do question for next branch. Player strength: Monster strength: Player strength is lower/higher
    //if statement lower/higher query respective branch
    if(PStrength > MStrength)
    
    
    else
    
    

    //recursive question for next branch. Player distance from base/monster. 
    if(DistanceFBase > DistanceFMonster)
    
    
    else
    
    
    //DECISION WOULD BE MADE
    //if statement?
    // inside query output decision?
    //cout <<  << 

        //QueryTree(node->NewBranch2);

        //MakeDecision(node);


//Step.....8ish?
void DecisionTree::Output()

    //take repsective node
    DisplayTree(m_RootNode);


//Step 9
void DecisionTree::DisplayTree(TreeNodes* CurrentNode)

    //if it doesn't exist, don't display of course
    if(CurrentNode == NULL)
    
        return;
    

    //////////////////////////////////////////////////////////////////////////////////////////////////
    //need to make a string to display for each branch
    cout << "Node ID " << CurrentNode->m_NodeID << "Decision Display: " << endl;

    //go down branch 1
    DisplayTree(CurrentNode->NewBranch1);

    //go down branch 2
    DisplayTree(CurrentNode->NewBranch2);


//Final step at least in this case. A way to Delete node from tree. Can't think of a way to use it yet but i know it's needed
void DecisionTree::RemoveNode(TreeNodes *node)

    //could probably even make it to where you delete a specific node by using it's ID
    if(node != NULL)
    
        if(node->NewBranch1 != NULL)
        
            RemoveNode(node->NewBranch1);
        

        if(node->NewBranch2 != NULL)
        
            RemoveNode(node->NewBranch2);
        

        cout << "Deleting Node" << node->m_NodeID << endl;

        //delete node from memory
        delete node;
        //reset node
        node = NULL;
    

TreeNodes.h

using namespace std;
//tree node class
class TreeNodes

public:
    //tree node functions
    TreeNodes(int nodeID/*, string QA*/);
    TreeNodes();

    virtual ~TreeNodes();

    int m_NodeID;

    TreeNodes* NewBranch1;
    TreeNodes* NewBranch2;
;

TreeNodes.cpp

//contrctor
TreeNodes::TreeNodes()

    NewBranch1 = NULL;
    NewBranch2 = NULL;

    m_NodeID = 0;


//deconstructor
TreeNodes::~TreeNodes()
 

//Step 3! Also step 7 hah!
TreeNodes::TreeNodes(int nodeID/*, string NQA*/)

    //create tree node with a specific node ID
    m_NodeID = nodeID;

    //reset nodes/make sure! that they are null. I wont have any funny business #s -_-
    NewBranch1 = NULL;
    NewBranch2 = NULL;

【问题讨论】:

好问题,所以我投了赞成票。根据您尝试实现的库(例如 castor),它允许您在 C++ 中本地实现一些逻辑编程范式,这可能会引起您的兴趣。 mpprogramming.com/cpp 天哪,有很多代码需要随机志愿者阅读...... 是的,但这确实表明我在某种意义上知道自己在做什么。我看到的许多问题都不足以解决,或者它们没有表明它们确实有效。我确实工作了!哈哈。我不希望很多人经历每一件小事,只是为了了解发生了什么。 而且在节目上看起来没那么大哈! searchAddNode 中没有您的“创建节点”代码。不管if.. 【参考方案1】:

如果我错了,请纠正我,但从http://dms.irb.hr/tutorial/tut_dtrees.php 和http://www.decisiontrees.net/?q=node/21 的图像来看,实际的决策逻辑应该在节点中,而不是在树中。您可以通过具有多态节点来建模,每个决策都有一个。通过对树结构进行一些更改并对决策委托进行少量修改,您的代码应该没问题。

【讨论】:

【参考方案2】:

基本上,您需要将所有内容分解为多个阶段,然后将您尝试实现的算法的每个部分模块化。

您可以使用函数/类和存根在伪代码或代码本身中执行此操作。

然后,您可以在某个函数中编写算法的每个部分,甚至在将它们全部加在一起之前测试该函数。您基本上会得到在算法实现中用于特定目的的各种函数或类。因此,在您构建树的情况下,您将拥有一个计算节点熵的函数,另一个将数据划分为每个节点的子集的函数,等等。

我在这里谈论的是一般情况,而不是专门针对决策树构造。如果您需要有关决策树算法和相关步骤的具体信息,请查看 Mitchell 的机器学习一书。

【讨论】:

【参考方案3】:

实现决策树的伪代码如下

createdecisiontree(data, attributes)

Select the attribute a with the highest information gain

for each value v of the attribute a

    Create subset of data where data.a.val==v ( call it data2)

    Remove the attribute a from the attribute list resulting in attribute_list2

    Call CreateDecisionTree(data2, attribute_list2)

您将不得不使用一些代码在每个级别存储节点,例如

决策树[attr][val]=new_node

【讨论】:

以上是关于C++ 决策树实现问题:在代码中思考的主要内容,如果未能解决你的问题,请参考以下文章

c++中基于大型决策树的人工智能设计模式

机器学习-------决策树算法

【理论篇】决策树剪枝策略

C++ 决策树存储

决策树特征重要性

机器学习 --决策树