• Flink配置文件


    单机测试时的配置:

    state.backend: filesystem
    # 取消的时候保存检查点
    execution.checkpointing.externalized-checkpoint-retention: RETAIN_ON_CANCELLATION
    # 60s 一次检查点
    execution.checkpointing.interval: 60s
    # 检查点语意
    execution.checkpointing.mode: EXACTLY_ONCE
    
    
    # Directory for checkpoints filesystem, when using any of the default bundled
    # state backends.
    #
    # state.checkpoints.dir: hdfs://namenode-host:port/flink-checkpoints
    state.checkpoints.dir: file:///tmp/flink1.12-checkpoints
    # Default target directory for savepoints, optional.
    #
    # state.savepoints.dir: hdfs://namenode-host:port/flink-savepoints
    state.savepoints.dir: file:///tmp/flink1.12-savepoints
    # Flag to enable/disable incremental checkpoints for backends that
    # support incremental checkpoints (like the RocksDB state backend). 
    #
    # state.backend.incremental: false
    
    # The failover strategy, i.e., how the job computation recovers from task failures.
    # Only restart tasks that may have been affected by the task failure, which typically includes
    # downstream tasks and potentially upstream tasks if their produced data is no longer available for consumption.
    
    jobmanager.execution.failover-strategy: region

    生产配置:

    # state.backend: filesystem
    state.backend: rocksdb
    state.backend.incremental: true
    # 以下目录每个集群节点都必须存在或有权自动创建
    state.backend.rocksdb.localdir: /var/data/flink-rosckdb
    # Directory for checkpoints filesystem, when using any of the default bundled
    # state backends.
    #
    state.checkpoints.dir: hdfs:///user/share/flink1.12/flink-checkpoints
    state.checkpoints.num-retained: 5
    # Default target directory for savepoints, optional.
    #
    state.savepoints.dir: hdfs:///user/share/flink1.12/flink-savepoints
    
    # Flag to enable/disable incremental checkpoints for backends that
    # support incremental checkpoints (like the RocksDB state backend). 
    #
    state.backend.incremental: true
    
    # The failover strategy, i.e., how the job computation recovers from task failures.
    # Only restart tasks that may have been affected by the task failure, which typically includes
    # downstream tasks and potentially upstream tasks if their produced data is no longer available for consumption.
    
    #jobmanager.execution.failover-strategy: region
    restart-strategy: fixed-delay
    restart-strategy.fixed-delay.attempts: 10
    restart-strategy.fixed-delay.delay: 30s
  • 相关阅读:
    上周热点回顾(7.29-8.4)团队
    云计算之路:AWS, Azure, Aliyun, UCloud提供的Windows操作系统团队
    上周热点回顾(7.22-7.28)团队
    我的MYSQL学习心得(推荐)
    深度学习笔记之使用Faster-Rcnn进行目标检测 (实践篇)
    深度学习笔记之使用Faster-Rcnn进行目标检测 (原理篇)
    深度学习笔记之基于R-CNN的物体检测
    深度学习笔记之目标检测算法系列(包括RCNN、Fast RCNN、Faster RCNN和SSD)
    深度学习笔记之神经网络、激活函数、目标函数和深度的初步认识
    深度学习笔记之CNN(卷积神经网络)基础
  • 原文地址:https://www.cnblogs.com/yoyowin/p/14751566.html
Copyright © 2020-2023  润新知