ruạṛ
3 ?L�i�o � " @ s d dl mZ d dlZd dlZejd,krzyd dlmZ W n ek rN eZY nX d dl Z d dl Z d dl m Z mZ d dlZn(d dlZd dlZd dlZd dl m Z mZ ejd-kr�d dlZnd dlZd dlZd dlZd dlZy d dlZW n ek �r d dlZY nX d dlZd dl Z d dl Z d dl Z d dl Z d dl Z d dl!Z!d dl"Z"d dl#Z#d dl$m%Z% y0d dl&m'Z'm(Z( d d l)m*Z*m+Z+ d d l,m-Z- W nB ek �r� d d l'm'Z' d d l.m(Z( d d l/m*Z*m+Z+m-Z- Y nX dZ0e j1j2e j1j3e4��Z5e j1j6dd�e j1j6dd�e j1j6dd�e j1j6e j1j2e5�d�e j1j6e j1j2e5�d�e j1j6e j1j2e5�d�e j1j3d�e j1j3d�e j1j3d�f Z7e j8dk�r�e j1j6e5ddd�e j1j6e5ddd�e j1j6e5ddd�fZ7dd� Z9d.dd�Z:dd � Z;d d � Z<d d � Z=d!d"� Z>d#d$� Z?g fd%d&�Z@G d'd(� d(�ZAd)d*� ZBeCd+k�reB� dS )/� )�print_functionN� )� basestring)�Queue�Empty� )� OptionParser)�urlparse� urlencode)�urlopen�Request)� HTTPError)r )r )r r r z1.3.1z/etcz agent360.inizagent360-custom.inizagent360-token.ini�ntz..�configc C sP t dd�} | j� }| jdd�}djdt ddj|� d | d | jjd d � f�S ) z� Return string with info about agent360: - version - plugins enabled - absolute path to plugin directory - server id from configuration file T)� dry_instance�enabled)�state� z Version: %szPlugins enabled: %sz, zPlugins directory: %sz Server: %s�agent�server)�Agent�_get_plugins_path� _get_plugins�join� __version__r �get)r � plugins_pathZplugins_enabled� r �:/root/tmp/pip-build-3x0y2zol/agent360/agent360/agent360.py�infoM s r �httpsc C s� t � }|jdddd� |jddtddd � |j� \}}|d }td d �}t|�d kr^|d }ntjjt d �}t|�dkr�|d }nd}|j d kr�d}n|j }|j d kr�djt � �} nd} d|kr�|j d�d } |j d�d }npytj� d } W n tk �r tj� } Y nX t| d |jjdd� d t|| ||| d��jd�d�j� j� } t| �dk�r�td| � t|d�jd || f � n td | � d S ) Nz-tz--tagsz Comma-separated list of tags)�helpz-az --automonr z6Enable/disable automatic monitoring of hosted websites)�type�defaultr! T)r � zagent360-token.ini� � �,�_z://�data�hello_api_hostz/hello)�user�hostname� unique_id�tags�domainszutf-8)r) � zGot server_id: %s�wz [DEFAULT] user=%s server=%s z Could not retrieve server_id: %s) r � add_option�int� parse_argsr �len�os�pathr �__FILEABSDIRNAME__r. Zautomon� _get_domains�split�uname�AttributeError�socket�getfqdnr r r r �encode�read�decode�print�open�write) �proto�parser�options�argsZuser_idr Ztoken_filenamer- r. r/ Z server_idr, r r r �hellob sN rI c C sl g } yNt jddg�}x:|j� j� D ]*}d|kr0q"|j� jd�}| j|d � q"W W n tk rf Y nX | S )NZ apachectlz-SZ namevhost� r )� subprocess� check_outputrA � splitlines�stripr: �append�FileNotFoundError)r/ �output�line�colsr r r �_get_apache_domains� s rT c C s� g } ynt jddg�}xZ|j� j� D ]J}d|kr0q"|j� jd�}t|�dkr"|d jdd�jd d�}| j|� q"W W n t k r� Y nX | S ) NZnginxz-TZ server_namerJ r% r$ �;r& �") rK rL rA rM rN r: r5 �replacerO rP )r/ rQ rR rS �domainr r r �_get_nginx_domains� s rY c C s g } yBt jdddg�}tj|�}x"|d d D ]}| j|d � q.W W n� tk r� y6t jdddd g�}x |j� j� D ]}| j|� qxW W nj tk r� xTtt t � t � ��D ]<}d |kr�q�|j d �r�q�|j d �r�q�|j d �r�q�| j|� q�W Y nX Y nX | S )NZwhmapi1z--output=jsonprettyZget_domain_infor) r/ rX Zplesk�binz--list�.z .localdomainz .localhostz.local) rK rL �json�loadsrO rP rA rM �list�setrT rY �endswith)r/ Zjson_str�responserX �strr r r r9 � s. r9 c C s t tt� �� d S )N)rB r5 r9 r r r r � count_domains� s rc c C s0 t | t�r&tjj| �}tjj|�d S | jS d S )Nr )� isinstancer r6 r7 �basename�splitext�__name__)�pluginre r r r � _plugin_name� s ri c 3 C s� t dd�}|j� }|tjkr*tjjd|� | sL|jdd�} tddj| � � �xZ| D �]P}td| � y,tjdkr�t j j |�}nt j |�\}}}W n. tk r� } ztd |� wTW Y d d }~X nX zny8tjdkr�t j j|�} |jj| � nt j||||�} W n0 tk �r, } ztd |� wTW Y d d }~X nX W d tjdk �rL|�rL|j� X y(| j� j|j�} ttj| d dd�� W qT tk �r� } ztd|� W Y d d }~X qTX qTW d S )z� Test specified plugins and print their data output after single check. If plugins list is empty test all enabled plugins. T)r r r )r z Check all enabled plugins: %sz, z%s:r r z Find error:Nz Load error:)�indent� sort_keyszExecution error:)r r )r r )r r )r r �sysr7 �insertr rB r � version_info� importlib�util� find_spec�imp� find_module� Exception�module_from_spec�loader� exec_module� load_module�close�Plugin�runr r\ �dumps) �pluginsr r � plugin_name�spec�fp�pathname� description�e�module�payloadr r r � test_plugins� s@ r� c @ s� e Zd Ze� Ze� Ze� Ze� ZdZd dd�Z dd� Z dd� Z dd � Z d d � Z d d � Zdd� Zdd� Zdd� Zdd� Zdd� Zd!dd�Zdd � Zd d � Zd S )"r Fc C s: | j � i | _|rdS | j� | j� | j� | j� dS )z0 Initialize internal strictures N)� _config_init� plugins_cache� _logging_init� _plugins_init�_data_worker_init� _dump_config)�selfr r r r �__init__" s zAgent.__init__c C s2 t jdkr t jj| jjdd��S | jjdd�S d S )Nr r r} )r6 �namer7 � expandvarsr r )r� r r r r 3 s zAgent._get_plugins_pathc C s� ddt jdddtjjtd�ddddddd d d dd �}d ddg}tjdkrTtj |�}n t j |�}|j t � || _ xF|D ]>}| j|� |dkr�| j j|dd� |d krt| j j|dd� qtW dS )z1 Initialize configuration object �< � �d r} �nor& zingest.monitoring360.iozapi.monitoring360.ioz/v2/server/pollz/var/log/agent360.log�a)� max_data_span� max_data_age� logging_level�threads�ttl�intervalr} r rK r+ r �api_hostr* �api_path�log_file� log_file_mode�max_cached_collectionsr � executionr) r r� r$ g �?NiX )r )�logging�INFOr6 r7 r r8 rl rn � configparser�RawConfigParser� ConfigParserr@ � ini_filesr �_config_section_creater_ )r� �defaults�sectionsr �sectionr r r r� : s> zAgent._config_initc C s | j j|�s| j j|� dS )zc Create an addition section in the configuration object if it's not exists N)r � has_section� add_section)r� r� r r r r� c s z Agent._config_section_createc C s� | j jdd�}tjdkr0tjj| j jdd��}n| j jdd�}| j jdd�}|dkrVn |dkrdd}n|d krrd}nd}|d kr�tj|d � n\ytj|||d d � W nD t k r� } z(tj|d � tj d|� tj d� W Y dd}~X nX tj d|� dS )z, Initialize logging faculty r r� r r� r� r1 r� �truncaterO �-)�levelz,%(asctime)-15s %(levelname)s %(message)s)�filename�filemoder� �formatz IOError: %szDrop logging to stderrNzAgent logging_level %i)r1 r� ) r �getintr6 r� r7 r� r r� � basicConfig�IOErrorr )r� r� r� r� r� r r r r� k s* zAgent._logging_initc C sr t jd� | j� }tjtjj|d��}|tjkr>tjjd|� i | _ �x&|D �] }t |�}|dkrdqL| j |� | j j |d�rL| j j |d�r�d| j |< qLtjd kr�tjj|�}ntj|�\}}}zpy8tjd kr�tjj|�} |jj| � ntj||||�} W n2 tk �r* d } t jd ttj� d �� Y nX W d tjdk �rJ|�rJ|j � X | �r^d| j | < qLt jd |� qLW d S )z& Discover the plugins r� z*.pyr r} r rK r r Nzimport_plugin_exception:%szimport_plugin:%s)r r )r r )r r ) r� r r �globr6 r7 r rl rm �scheduleri r� r � getbooleanrn ro rp rq rr rs ru rv rw rx rt �errorrb �exc_infory ) r� r � filenamesr� r� r r� r� r� r� r r r r� � s> " zAgent._plugins_initc C s( t jtj|ft jt jdd�}tjdtj� |j � | j j dd�}t |�}| j j |d�}|| p^d}|j � xB|jdkr�|d kr�tjd tj� |� tj|� |d8 }|j � qjW |jdkr�tjd tj� |j � tj|j tj� |j� \}}|jd ks�|�rtjd tj� ||j|� |�r tj|�} nd} | S ) z0 Execute /task/ in a subprocess T)�stdout�stderr�universal_newlinesz %s:process:%ir� r� r� r$ Nr z %s:tick:%iz %s:kill:%iz %s:%s:%s:%s)rK �Popenrl � executable�PIPEr� �debug� threading� currentThread�pidr r� ri �poll� returncode�time�sleepr� r6 �kill�signal�SIGTERM� communicate�pickler] ) r� �task�processr� r� r� Zticksr� r� �retr r r �_subprocess_execution� s2 zAgent._subprocess_executionc C sb �x@| j r tjdtj� � P tjdtj� | jj� � y| jj� }W n t k rX P Y nX tjdtj� |� t |�}y| j j |d�}W n d}Y nX t j � }t|t�r�| j|�}nry<| jj|| jj |g �i� |j| j| d�}|j| j �}W n4 tk �r& tjd� dttj� d �i}Y nX | jj|||||d �� qW | j jtj� � | j j � d S ) zW Take queued execution requests, execute plugins and queue the results z %s:shutdownz%s:exec_queue:%iz %s:task:%sr� r� )Z agent_cacheZplugin_exception� exceptionr )�tsr� r� r� r� N) �shutdownr� r r� r� r� �execute�qsize� get_nowaitr ri r r r� rd r r� r� �updaterz r{ rt r� rb rl r� �metrics�put�cemetery�hire�release)r� r� r� r� r� r� rh r r r � _execution� sD zAgent._executionc C s t jdtj� � | jjdd�}| jjdd�}| jjdd�}| jjdd�}| jjdd�}| jjdd �}| jjdd �}| jjdd �}g } g } d } �x�| r�d }n| jjdd�}tj� } | jr�t jdtj� � P t j dtj� | j j � t | �� xX| j j � �r>y| j | j j� � W q� tk �r: } zt jd| � W Y dd} ~ X q�X q�W | �r�tdd� | D ��}tdd� | D ��}tj� }d}|| |k�r�t j d� d }d}n || |k�r�t jd� d }d }|�r�d} dd||f d�}t j dtj| d d d �� |�o�|�s t jd � d }�n�zԐy@tjtj� d �}tjd0k�rBt j j ||d!d"�}nt j ||d!d"�}| �r�t jd#t | �� xj| �r�|j d$d%|t!f | d& |d'� |j"� }|j#� |j$d(k�r�| d&= t j d)|j$� nt%d*|j$ ���qjW t jd+� |j d$d%|t!f t&j't(tj| �d, �j)� �|d'� |j"� }|j#� |j$d(k�r@t j d)|j$� d }nt%d*|j$ ��W n� tk �r� } znt jd-| � |d&k�r�t | �|k�r�| d&= t jd.|� t jd/� | j t&j't(tj| �d, �j)� �� g } W Y dd} ~ X nX W d|j*� X |�r�g } |tj� | }|d&kr�tj+|� q�W dS )1zA Take and collect data, send and clean if needed z%sr) r� r� r r� r� r r+ r� r� Tr� z %s:shutdownz %s:data_queue:%i:collection:%izData queue error: %sNc s s | ]}|d V qdS )r� Nr )�.0r� r r r � <genexpr>' s z Agent._data.<locals>.<genexpr>c s s | ]}|d V qdS )r� Nr )r� r� r r r r� ( s Fz Max data spanz Max data agezapplication/jsonz ApiKey %s:%s)z Content-type� Authorizationzcollection: %sr% )rj rk z&Empty server or user, nowhere to send.)�cafiler � )�context�timeoutz Sending cached collections: %i�PUTz %s?version=%sr )�headers�� zSuccessful response: %szUnsuccessful response: %szAll cached collections sentr z Failed to submit collection: %szCReach max_cached_collections (%s): oldest cached collection droppedz,Cache current collection to resend next time)r ),r� r r� r� r r r� r� r� r� r) r� r5 rO r� rt r� �min�max�warningr\ r| �ssl�create_default_context�certifi�whererl rn �http�client�HTTPSConnection�httplib�requestr � getresponser@ �status� ValueError�bz2�compressrb r? ry r� )r� r� r� Zmax_ageZmax_spanr r+ r� r� Zcached_collectionsZ collectionZ initial_dataZloop_tsr� Zfirst_tsZlast_ts�now�send�cleanr� �ctx� connectionra �sleep_intervalr r r �_data s� $ " z Agent._datac C s t jd� tj| jd�j� dS )z/ Initialize data worker thread r� )�targetN)r� r r� �Threadr �start)r� r r r r� x s zAgent._data_worker_initc C s< t jdkrtj� }ntj� }| jj|� tjd|j� � dS )z, Dumps configuration object r z Config: %sN)r ) rl rn �io�StringIOr rD r� r �getvalue)r� �bufr r r r� s zAgent._dump_configr c C s� | j � }g }xztjtjj|d��D ]b}t|�}|dkr8q"| j|� |dkrd| jj|d�r�|j |� q"|dkr"| jj|d�s"|j |� q"W |S )z0 Return list with plugins names z*.pyr} r �disabled) r r� r6 r7 r ri r� r r� rO )r� r r r} r� r~ r r r r � s zAgent._get_pluginsc C sV t jd| jj� � x>y| jj� }W n tk r8 P Y nX t jd|� |j� qW dS )zb Join with dead workers Workaround for https://bugs.python.org/issue37788 z cemetery:%iz joining:%sN)r� r� r� r� r� r r )r� �threadr r r �_rip� s z Agent._ripc s� t jd� | jjdd�}tj| jjdd��| _�y��x�| j� t j � � t j dtj � � xx| j j � r�| j j� }|d }t j d|� |jd �}|r�t� �| jj|d� | j|< t|tj�r�|j|d < | jj|� qXW � fd d �| jj� D �}x�|D ]�}t j d |� | j|= | jj|� | jjd ��rzy&tj| j d�}|j � t j d|� W n0 t k �rv } zt j d|� W Y dd}~X nX q�t j d� | j j� dd| jjdd�id�� q�W dt j � � }|dk�r�t j |� q6t j d� t j d� q6W W n� t!k �r� t j t"j#� d � t jd� | j� d} x�| �r�tj$� } t jd | � d d � | D �} | �sht jd � t"j%d� d| _&t jd t'| �� x&| D ] }t jd ||� |j(|� �q�W �q&W Y n2 t k �r� } zt j)d!| � W Y dd}~X nX dS )"z. Start all the worker threads zAgent main loopr r� r� r� z %i threadsr� z metrics:%sr� c s g | ]\}}|� kr|�qS r r )r� �what�when)r� r r � <listcomp>� s z Agent.run.<locals>.<listcomp>z scheduling:%sF)r z new_execution_worker_thread:%sz Can not start new thread: %sNZthreads_cappedZagent_internalZthreads_capping)r� r� r� g �?r z'not enough time to start worker threadsg�������?z Shutting downTzRemaining threads: %sc S s( g | ] }|j � rt|tj� r|�qS r )�isDaemonrd r� � _MainThread)r� r r r r r � s zBye!z Waiting for %i threads to exitzJoining with %s/%fzWorker error: %s)*r� r r �getfloatr� � Semaphorer� r� r r� r� � activeCountr� r� r� r r3 r� rd �types� ModuleType�__file__r) r� �itemsr� �acquirer r� r rt r� r� �KeyboardInterruptrl r� � enumerate�exitr� r5 r r� ) r� r� r� r� rh r� r r� r �wait_forZ all_threadsr )r� r r{ � s| z Agent.runN)F)r )rg � __module__� __qualname__r r� r� r) r� r� r� r r� r� r� r� r� r� r r� r� r r r{ r r r r r s& )!, /r r c C sl t tj�dk�r^tjd jd�r8tjd dd � tjd< tjd dkr^tdjd�� tj� n�tjd d kr�tt� � tj� n�tjd d kr�tt� tj� n�tjd d kr�tjd= tjt � � n�tjd d kr�tjd= tjt � � nttjd d k�rtjd= tjt dd�� nJtjd dk�r<tjt tjdd � �� n tdtjd tj d� tjd� n t � j� d S )Nr$ z--r% r! r �!Run without options to run agent.�,Acceptable options (leading -- is optional):�4 help, info, version, hello, insecure-hello, testr �versionrI z count-domainszinsecure-hellor� )rE �testzInvalid option:)�file)r r! r" )r5 rl �argv� startswithrB r r r r rI rc r� r� r r{ r r r r �main s: r( �__main__)r )r r )r )D� __future__r r� rl rn Z past.builtinsr � ImportErrorrb r� � http.clientr� �queuer r r r� r� r �importlib.utilro rr r� r� r� r\ Z simplejsonr� r6 r� r� r= rK r� r� r �optparser � urllib.parser r �urllib.requestr r � urllib.errorr �urllib�urllib2r r7 �dirname�abspathr r8 r r� r� r rI rT rY r9 rc ri r� r r( rg r r r r �<module> s� 8! 0 i$
cải xoăn