big data science(course notes)

Posted 2020-10-08 satyrs

1 大数据：batch，interactive query，streaming

2 集群环境有三大挑战:分别是并行化、单点失败处理、资源共享。采用以并行化的方式重写应用程序、对单点失败的处理方式、动态地进行计算资源的分配等解决方案

3 

address of any protocol control  blocks associated with sockets and the flow hash

sockets used by server processes

protocol-cloned

           routes (routes generated by a RTF_PRCLONING parent route)

 address_family

           Limit statistics or address control block reports to those of the

           specified address family.  The following address families are rec-

           ognized: inet, for AF_INET, inet6, for AF_INET6 and unix, for

           AF_UNIX.

 

merges the values for each key  

//
def reduceByKey(partitioner:Partitioner,func:(V,V)=>V): RDD[(K,V)] = self.withScope{
    combineByKeyWithClassTag[V]((v:V)=>v, func, func, partitioner)
}