如何在ANN中设置偏差并将Sigmoid更改为ReLU功能？

Question

我正在尝试通过人工神经网络创建数据预测模型。以下代码是通过许多书籍创建的基于Python的ANN代码的一部分。此外，预测值和实际值之间的错误率不满足低于19％。我试图增加隐藏层的数量，但它并没有极大地影响错误率。我认为这可能是Sigmoid功能的限制而不是考虑Bias。我环顾了一个月，发现了如何构建ReLU和Bias，但我找不到Bias和ReLU的范围。

Q1 =如何将Sigmoid转换为ReLU和Q2 =如何将偏差添加到我的代码？

Q3 =此外，如果我将Sigmoid更改为ReLU，我是否必须使我的数据集为0.0~1.0范围？这是因为Sigmoid函数接受0.0~1.0范围的数据，但我不知道ReLU允许的范围。

我很遗憾地提出一个基本问题。

class neuralNetwork:
# initialize the neural network
def __init__(self, input_nodes, hidden_nodes, output_nodes, learning_rate):

#
    self.inodes = input_nodes
    self.hnodes = hidden_nodes
    self.onodes = output_nodes

    # link weight matrices, wih and who
    self.wih = numpy.random.normal(0.0, pow(self.hnodes, -0.5), (self.hnodes, self.inodes))
    self.who = numpy.random.normal(0.0, pow(self.onodes, -0.5), (self.onodes, self.hnodes))

    # learning rate
    self.lr = learning_rate

    # activation function is the sigmoid function
    self.activation_function = lambda x: scipy.special.expit(x)

    pass

# train the neural network
def train(self, inputs_list, targets_list):
    # convert inputs list to 2d array
    inputs = numpy.array(inputs_list, ndmin=2).T
    targets = numpy.array(targets_list, ndmin=2).T

    # calculate signals into hidden layer
    hidden_inputs = numpy.dot(self.wih, inputs)
    # calculate the signals emerging from hidden layer
    hidden_outputs = self.activation_function(hidden_inputs)

    # calculate signals into final output layer
    final_inputs = numpy.dot(self.who, hidden_outputs)
    # calculate the signals emerging from final output layer
    final_outputs = self.activation_function(final_inputs)

    # output layer error is the (target - actual)
    output_errors = targets - final_outputs
    # hidden layer error is the output_errors, split by weights, recombined at hidden nodes
    hidden_errors = numpy.dot(self.who.T, output_errors)

    # update the weights for the links between the hidden and output layers
    self.who += self.lr*numpy.dot((output_errors*final_outputs*(1.0-final_outputs)), numpy.transpose(hidden_outputs))
    # update the weights for the links between the input and output layers
    self.wih += self.lr*numpy.dot((hidden_errors*hidden_outputs*(1.0-hidden_outputs)), numpy.transpose(inputs))

    pass

# query the neural network
def query(self, inputs_list) :

    inputs = numpy.array(inputs_list, ndmin=2).T

    # convert hidden list to 2d array
    hidden_inputs = numpy.dot(self.wih, inputs)
    # calculate signals into hidden layer
    hidden_outputs = self.activation_function(hidden_inputs)


    final_inputs = numpy.dot(self.who, hidden_outputs)
    final_outputs = self.activation_function(final_inputs)
    return final_outputs        
    pass