本文运行流程介绍来自jumpserver版本号:v2.23.2
入口文件 run_server.py
run_server中通过subprocess.call,用python3运行了同级目录下jms,并传入参数 start all,进入jms.py
首先配置BASE_DIR和APP_DIR全局变量,设置DJANGO_SETTINGS_MODULE值为jumpserver.settings后调用django.steup(),这里就不说了,djang.steup()源码流程可看这里,然后配置logging格式以及必要导入成功检测,并初始化logging对象和创建static和media文件夹,准备工作第一步完成。源码如下:
BASE_DIR = os.path.dirname(os.path.abspath(__file__)) APP_DIR = os.path.join(BASE_DIR, 'apps') os.chdir(APP_DIR) sys.path.insert(0, APP_DIR) os.environ.setdefault("DJANGO_SETTINGS_MODULE", "jumpserver.settings") django.setup() logging.basicConfig(level=logging.DEBUG, format="%(asctime)s %(message)s", datefmt="%Y-%m-%d %H:%M:%S") try: from jumpserver import const __version__ = const.VERSION except ImportError as e: print("Not found __version__: {}".format(e)) print("Python is: ") logging.info(sys.executable) __version__ = 'Unknown' sys.exit(1) try: from jumpserver.const import CONFIG from common.utils.file import download_file except ImportError as e: print("Import error: {}".format(e)) print("Could not find config file, `cp config_example.yml config.yml`") sys.exit(1) os.environ["PYTHONIOENCODING"] = "UTF-8" logging.basicConfig( format='%(asctime)s %(message)s', level=logging.INFO, datefmt='%Y-%m-%d %H:%M:%S' ) logger = logging.getLogger() try: os.makedirs(os.path.join(BASE_DIR, "data", "static")) os.makedirs(os.path.join(BASE_DIR, "data", "media")) except: pass
然后到文件最后,创建parse,并准备参数配置,格式化参数生产参数对象args(注:这里的args为全局变量,所以jms中的函数可以直接使用该变量,不用当参数传递),args.action为"start",因此走start_services,该部分源码如下:
if __name__ == '__main__': parser = argparse.ArgumentParser( description=""" Jumpserver service control tools; Example: \r\n %(prog)s start all -d; """ ) parser.add_argument( 'action', type=str, choices=("start", "stop", "restart", "status", "upgrade_db", "collect_static"), help="Action to run" ) parser.add_argument( "services", type=str, default='all', nargs="*", choices=("all", "web", "task"), help="The service to start", ) parser.add_argument('-d', '--daemon', nargs="?", const=True) parser.add_argument('-w', '--worker', type=int, nargs="?", default=4) parser.add_argument('-f', '--force', nargs="?", const=True) args = parser.parse_args() action = args.action if action == "upgrade_db": upgrade_db() elif action == "collect_static": collect_static() else: start_services()
start_services源码如下:
def start_services(): services = args.services if isinstance(args.services, list) else [args.services] if action == 'start' and {'all', 'web'} & set(services): prepare() start_args = [] if args.daemon: start_args.append('--daemon') if args.worker: start_args.extend(['--worker', str(args.worker)]) if args.force: start_args.append('--force') try: management.call_command(action, *services, *start_args) except KeyboardInterrupt: logging.info('Cancel ...') time.sleep(2) except Exception as exc: logging.error("Start service error {}: {}".format(services, exc)) time.sleep(2)
services为"all",因此会执行prepare()函数,该函数中分别执行:1.调用django接口检查数据库连接是否正常 2.收集静态文件和数据库迁移 3.清除缓存 4.下载两个ip库,源码较多也简单,就不贴了。最后通过django接口management.call_command调用start并传入all、--daemon、--worker 4、--force参数,这里讲下management.call_command,源码精简贴下,不重要的就不贴了,如下:
def call_command(command_name, *args, **options): """ Call the given command, with the given options and args/kwargs. This is the primary API you should use for calling specific commands. `command_name` may be a string or a command object. Using a string is preferred unless the command object is required for further processing or testing. Some examples: call_command('migrate') call_command('shell', plain=True) call_command('sqlmigrate', 'myapp') from django.core.management.commands import flush cmd = flush.Command() call_command(cmd, verbosity=0, interactive=False) # Do something with cmd ... """ if isinstance(command_name, BaseCommand): # Command object passed in. command = command_name command_name = command.__class__.__module__.split('.')[-1] else: # Load the command object by name. try: app_name = get_commands()[command_name] except KeyError: raise CommandError("Unknown command: %r" % command_name) if isinstance(app_name, BaseCommand): # If the command is already loaded, use it directly. command = app_name else: command = load_command_class(app_name, command_name) # 中间省略 。。。 return command.execute(*args, **defaults)
首先command_name为"start"字符串,走else,再看app_name = get_commands()[command_name],get_commands()在django.steup()同篇文章django runserver源码中有讲,其实质做的是以字典方式保存django/core/management/commands和所有子app目录/management/commands目录下所有以非"_"开头模块文件名称和app字符串的对应关系,在项目目录下apps/common/management/commands下有start.py文件,因此此时app_name为“common",最后走load_command_class(app_name, command_name),load_command_class源码如下:
def load_command_class(app_name, name): """ Given a command name and an application name, return the Command class instance. Allow all errors raised by the import process (ImportError, AttributeError) to propagate. """ module = import_module('%s.management.commands.%s' % (app_name, name)) return module.Command()
回到call_command函数,此时command为start.py中的Command类(一下简称Command类)的实力对象,最后调用对象的execute()方法,Command类继承BaseActionCommand类,BaseActionCommand类未定义execute方法,于是到BaseCommand类的execute方法,不重要的也不贴了,源码如下:
def execute(self, *args, **options): """ Try to execute this command, performing system checks if needed (as controlled by the ``requires_system_checks`` attribute, except if force-skipped). """ # 前面省略。。。 output = self.handle(*args, **options) if output: if self.output_transaction: connection = connections[options.get('database', DEFAULT_DB_ALIAS)] output = '%s\n%s\n%s' % ( self.style.SQL_KEYWORD(connection.ops.start_transaction_sql()), output, self.style.SQL_KEYWORD(connection.ops.end_transaction_sql()), ) self.stdout.write(output) return output
最终调用self.handle()方法,Command类下未定义该方法,于是到BaseActionCommand类,该类handle方法源码如下:
def handle(self, *args, **options): self.initial_util(*args, **options) assert self.action in Action.values, f'The action {self.action} is not in the optional list' _handle = getattr(self, f'_handle_{self.action}', lambda: None) _handle()
initial_util方法中创建ServicesUtil类的实例属性,并赋值为self.util。创建ServicesUtil类时传入参数分别为:run_daemon:True,stop_daemon:False,force_stop:True。services流程讲一下,源码先贴一下:
@classmethod def get_service_objects(cls, service_names, **kwargs): services = set() for name in service_names: method_name = f'{name}_services' if hasattr(cls, method_name): _services = getattr(cls, method_name)() elif hasattr(cls, name): _services = [getattr(cls, name)] else: continue services.update(set(_services)) service_objects = [] for s in services: service_class = cls.get_service_object_class(s.value) if not service_class: continue kwargs.update({ 'name': s.value }) service_object = service_class(**kwargs) service_objects.append(service_object) return service_objects
传入的service_names为["all“],因此_services为[cls.gunicorn, cls.daphne, cls.flower, cls.celery_ansible, cls.celery_default, cls.beat],函数内的services为{cls.gunicorn, cls.daphne, cls.flower, cls.celery_ansible, cls.celery_default, cls.beat},service_objects(返回的services)为services.GunicornService、services.DaphneService、services.FlowerService、services.CeleryDefaultService、services.CeleryAnsibleService、services.BeatService这六个类的实例对象组成的列表。回到handle方法,_handle为self._handle_start(),最后调用该方法,该方法内调用self.util.start_and_watch()方法执行,于是到ServicesUtil.start_and_watch方法,源码如下:
def start_and_watch(self): logging.info(time.ctime()) logging.info(f'JumpServer version {__version__}, more see https://www.jumpserver.org') self.start() if self.run_daemon: self.show_status() with self.daemon_context: self.watch() else: self.watch()
先看self.start(),遍历self._services,也就是遍历上述6个类的实例对象,并调用其start方法,这6个类都继承自BaseService类(jumpserver团队自己定义的服务接口类),且6各类均未定义start方法,于是来到BaseService类的start方法,源码如下:
def start(self): if self.is_running: self.show_status() return self.remove_pid() self.open_subprocess() self.write_pid() self.start_other()
self.is_running是通过对os.kill(self.pid, 0)是否异常来判断该进程号是否在运行,最开始肯定是未运行状态,于是走self.remove_pid(),该方法是通过os.unlink删除tmp文件夹下对应服务的pid文件,然后走self.open_subprocess(),通过subprocess.Popen调用self.cmd,并传入self.cwd和输出文件配置,这里就是真正的开启每个服务了,具体如何开启可以看六个类的cmd属性(property)。然后通过self.write_pid写pid文件,self.start_other是留的一个空函数。回到ServicesUtil.start_and_watch方法,首先self.run_daemon为True,因此调用self.show_status()打印六个服务每个服务是running还是stoped,最后在守护进程上下文内执行self.watch(),self.watch中每隔30秒执行一次self._watch(),self._watch()中是执行六个服务类实例对象的watch方法,服务类对象的watch方法中检查服务是否在运行,如果是未运行状态则重新运行,如果self.watch中捕获到KeyboardInterrupt(键盘退出)异常,则调用self.clean_up()进行清理操作,退出程序。贴下ServicesUtil类的watch()方法,如下:
# -- watch -- def watch(self): while not self.EXIT_EVENT.is_set(): try: _exit = self._watch() if _exit: break time.sleep(self.check_interval) except KeyboardInterrupt: print('Start stop services') break self.clean_up()