
­­­­­­­­­­­­­­­­­­
<!DOCTYPE html>
<html>
3
ie\                @   s  d dl Z d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl	Z	d dl
Z
d dlZd dlZd dlZd dlZd dlZd dlZd dlmZmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZmZ d d	lmZ d dlZyd dl Z W n e!k
r   Y nX d dl"Z"d dl#Z"d
dl$m%Z%m&Z&m'Z' d
dl(m)Z)m*Z*m+Z+m,Z, d
dl-m.Z.m/Z/m0Z0 d
dl1m2Z2m3Z3 d
dl4T d
dl5m6Z6m7Z7 d
dl8m9Z9m:Z:m;Z;m<Z<m=Z=m>Z> d
dl?m@Z@mAZA d
dlBmCZC d
dl&mDZDmEZE d
dl&mFZFmGZGmHZH d
dl&mIZImJZJmKZK d
dl&mLZL d
dl&mMZM d
dl&mNZN d
dlOmPZPmQZQ d
dlRmSZSmTZTmUZU d
dlVmWZW d
dlXmYZY d
dlZm[Z[ d
dl\m]Z]m^Z^ d
d l_m`Z` d!d"l$maZambZb d!d#l$mcZcmdZdmeZemfZf d!d$l$mgZgmhZhmiZimjZjmkZk d!d%l'mlZl d!d&lmmnZn d!d'l$moZo ejpjqejpjrejpjsetd(Zudd)dd*d)d+d,d-Zvd.d/ Zwyevd0d1d2d3 d4d5gZxW n eyk
rj   d4gZxY nX ejzexd6d7d8 Z{d9d: Z|d;Z}ej~jejpje} d<e} d=d>d? ZG d@dA dAecZG dBdC dCeZejd5exkdDG dEdF dFeZG dGdH dHeZG dIdJ dJeZG dKdL dLeZG dMdN dNeZG dOdP dPeZG dQdR dReZdSdT ZdUdV ZG dWdX dXZG dYdZ dZZd[d\ Zd]d^ Zej~jd_ee j d`da Zej~jdbee.jj dcdd ZdS )e    N)	unhexlify
b2a_base64)ConfigParser)datetime)timezone)	timedelta)sha256)BytesIOStringIO)patch   )xattrhelpersplatform)ArchiveChunkBufferflags_noatimeflags_normal)Archiverparse_storage_quotaPURE_PYTHON_MSGPACK_WARNING)Cache
LocalCache)*)bytes_to_longnum_aes_blocks)KeyfileKeyBaseRepoKey
KeyfileKey
PassphraseTAMRequiredErrorArchiveTAMRequiredError)RepoIdMismatchNotABorgKeyFile)FileIntegrityError)Locationget_security_dir)ManifestMandatoryFeatureUnsupportedArchiveInfo)EXIT_SUCCESSEXIT_WARNING
EXIT_ERROR)
bin_to_hex)MAX_S)msgpack)RstToTextLazyrst_to_terminal)	IECommandPatternMatcherparse_pattern)Item)
LockFailed)setup_logging)RemoteRepositoryPathNotAllowed)
Repository   )has_lchflags
has_llfuse)BaseTestCase	changedirenvironment_variable
no_selinux)are_symlinks_supportedare_hardlinks_supportedare_fifos_supportedis_utime_fully_supportedis_birthtime_fully_supported)fakeroot_detected)make_attic_repo)keyz..F    )archiverforkexeinputbinary_outputc             O   s  |ryX|d krt jddf}n$t|tr.|f}nt|ts@tdtj|| tj|d}d}	W nZ tj	k
r }
 z|
j
}|
j}	W Y d d }
~
X n, tk
r }
 zd}|
j}	W Y d d }
~
X nX |r|	|fS |	tj|fS nt jt jt j  }}}zt|j t _t|t j_t }tj|dd } t _t _| d kr8t } d	d
 | _t| _tt_y| jt|}W nF tk
r }
 z(|j   |
j|r|j! n
|j! j fS d }
~
X nX | j"|}	|j   |	|r|j! n
|j! j fS |||  t _t _t _X d S )Nz-mzborg.archiverz"exe must be None, a tuple or a str)stderrrN   r    zutf-8)encodingc              W   s   d S )N )argsrS   rS    /usr/lib64/python3.6/archiver.py<lambda>d   s    zexec_cmd.<locals>.<lambda>)#sys
executable
isinstancestrtuple
ValueError
subprocessZcheck_outputZSTDOUTZCalledProcessErroroutput
returncode
SystemExitcodeosfsdecodestdinstdoutrP   r
   decoder	   bufferioTextIOWrapperr   Zprerun_checksr*   	exit_coder   
parse_argslistflushgetvalueZrun)rK   rL   rM   rN   rO   rT   kwborgr^   reterd   re   rP   Zoutput_textrS   rS   rU   exec_cmdE   sN    



,
rs   c              C   s6   t jdsdS tjddgtjd} | j \}}d|kS )NtarFz	--version)re   s   GNU tar)shutilwhichr]   PopenPIPEZcommunicate)popenre   rP   rS   rS   rU   have_gnutarv   s
    
rz   helpzborg.exeT)rM   rL   pythonbinary)Zparamsc                s8   | j dkrd  n| j dkr d ntd fdd}|S )Nr|   r}   zborg.exez"param must be 'python' or 'binary'c                 s   t |  dd|S )NT)rM   rL   )rs   )rT   ro   )rM   rS   rU   exec_fn   s    zcmd.<locals>.exec_fn)Zparamr\   )Zrequestr~   rS   )rM   rU   cmd   s    

r   c             C   s   |j d}|j d}|j d}|jdjd | dddt| \}}|tksRt| d	d
| t|\}}|tksvttt|$ | dd
| \}}|tkstW d Q R X | dd
| d\}}|tkst| d	d
| t|\}}|tkstd S )NreporN   r^   Z	test_filecontentinitz--encryption=nonez%screatez%s::archiveextractzdoes/not/match)	mkdirjoinwriterZ   r*   AssertionErrorr?   r+   r,   )r   tmpdirr   rN   r^   rcoutrS   rS   rU   test_return_codes   s    


r   z/tmp/borg-mountzneeds a 16MB fs mounted on %s)reasonc          )   C   s
  ddd}t dd t}tjj|s,ttjj|d}tjj|d}tjj|d}xtd	D ]}tj	|dd
 tj	|dd
 ||dddd | d|\}}|t
krtd|| |t
kstzd\}	}
x|	r|
d7 }
y||dd W n6 tk
r" } z|jtjkrP  W Y d d }~X nX z6| dd||
f |\}}|t
k}	|	sXtd|| W d tj	tjj|ddd
 tjtjj|d X qW W d tj	|dd
 X | d|\}}|t
krtd|| | dd|\}}|t
krtd|| |t
ksbtqbW W d Q R X d S )NTc          
   S   s   t j| dd tj|  |r<tjd|}|dkr<tjd|}xLt|D ]@}tjj| d| }t	|d}tj
|}|j| W d Q R X qFW d S )NT)ignore_errorsr;   zfile%03dwb)ru   rmtreerb   r   randomZrandintrangepathr   openurandomr   )dircountsizerndifnfdatarS   rS   rU   
make_files   s    

z"test_disk_full.<locals>.make_filesYES)!BORG_CHECK_I_KNOW_WHAT_I_AM_DOINGr   rN   reserved   )r   P   i F)r   r   r   r;      i@ r   z%s::test%03dzlock.exclusivezlock.rosterrl   checkz--repair)T)Tr   )r@   DF_MOUNTrb   r   existsr   r   r   ru   r   r*   printOSErrorerrnoZENOSPCremove)r   r   mountr   rN   r   jr   r   successr   errrS   rS   rU   test_disk_full   sP    


r   c               @   sZ   e Zd ZdZdZdZdd Zdd Zdd	 Zd
d Z	dd Z
dd ZdddZdd ZdS )ArchiverTestCaseBaseNFrQ   c             C   s  dt jd< dt jd< dt jd< dt jd< | j r6t p8d | _tj | _t jj	| jd| _
| j| j
 | _t jj	| jd	| _t jj	| jd
| _t jj	| jd| _t jj	| jd| _t jj	| jd| _t jj	| jd| _| jt jd< | jt jd< t j| j t j| jd t j| j t j| j t j| j t| jd}|jd W d Q R X t| jd}|jd W d Q R X t j | _t j| j d S )Nr   r   "BORG_DELETE_I_KNOW_WHAT_I_AM_DOINGZwaytooeasyonlyfortestsBORG_PASSPHRASEZdisabledZBORG_SELFTEST
repositoryrN   r^   keyscacheZexcludespatternsZBORG_KEYS_DIRZBORG_CACHE_DIRi  r   s1   input/file2
# A comment line, then a blank line

sI   +input/file_important
- input/file*
# A comment line, then a blank line

)rb   environFORK_DEFAULTr   rK   tempfileZmkdtempr   r   r   repository_pathprefixrepository_location
input_pathoutput_path	keys_path
cache_pathexclude_file_pathpatterns_file_pathr   chmodr   r   getcwd_old_wdchdir)selffdrS   rS   rU   setUp   s6    





zArchiverTestCaseBase.setUpc             C   s&   t j| j tj| jdd t  d S )NT)r   )rb   r   r   ru   r   r   r7   )r   rS   rS   rU   tearDown  s    zArchiverTestCaseBase.tearDownc       	         s   |j dd}|j dd }|jdd}|d kr2| j}t||| j| jd|\}}||kr`t| | j|| |rxtj	 nt |rdnd}|j
 fd	d
|jddD }|S )Nrj   r   rL   rO   F)rL   rM   rK   rJ   rQ   c             3   s   | ]} |kr|V  qd S )NrS   ).0line)pp_msgrS   rU   	<genexpr>,  s    z+ArchiverTestCaseBase.cmd.<locals>.<genexpr>T)keepends)popgetr   rs   EXErK   r   assert_equalr   encoder   
splitlines)	r   rT   ro   rj   rL   rO   rq   r^   emptyrS   )r   rU   r     s     zArchiverTestCaseBase.cmdc             C   s   | j dd| jd | t d S )Nr   z--compression=lz4z::)r   r   src_dir)r   namerS   rS   rU   create_src_archive0  s    z'ArchiverTestCaseBase.create_src_archivec             C   sF   t | jdd}|& tj|tj\}}t||||}W d Q R X ||fS )NT)	exclusive)r:   r   r'   loadNO_OPERATION_CHECKr   )r   r   r   manifestrI   archiverS   rS   rU   open_archive3  s
    z!ArchiverTestCaseBase.open_archivec             C   s   t | jddS )NT)r   )r:   r   )r   rS   rS   rU   open_repository:  s    z$ArchiverTestCaseBase.open_repositoryr   c          	   C   s   |dko|ot ||k s"tdtjj| j|}tjjtjj|sXtjtjj| t	|d }|d krtd| }|j
| W d Q R X d S )Nr   zsize and contents do not matchr      X)lenr   rb   r   r   r   r   dirnamemakedirsr   r   )r   r   r   contentsfilenamer   rS   rS   rU   create_regular_file=  s    "z(ArchiverTestCaseBase.create_regular_filec             C   s  | j dd#d | j ddd | j dd$d tjdd t r`tjtjj| jdtjj| jd	 t r~tj	d
tjj| jd | j ddd t
j rt
j| jrt
jtjj| jddd t
jtjj| jddd t rtjtjj| jd trtjtjj| jdtj yZtjddtjB tjdd tjddtjB tjdd tjdd tjddd d}W nV tk
r   d}Y n> tk
r } z |jtjtjfkr d}W Y d d }~X nX tj d | j d!d"d |S )%zFCreate a minimal test case including all supported file types
        file1i   r   )r   flagfilez
dir2/file2zinput/file1i	  hardlink	somewherelink1	fusexattrr;   zuser.foos   barz
user.emptyrJ   fifo1z
input/bdevi  
   r   z
input/cdev   (   z
input/dir2im  r      TFNr   r   i @ i @ )!r   rb   r   rC   linkr   r   r   rB   symlinkr   XATTR_FAKEROOT
is_enabledsetxattrrD   mkfifor<   r   	set_flagsstat	UF_NODUMPmknodS_IFBLKmakedevS_IFCHRchownPermissionErrorr   r   ZEINVALZENOSYStimesleep)r   	have_rootrr   rS   rS   rU   create_test_filesG  s@    
z&ArchiverTestCaseBase.create_test_files)r   N)__name__
__module____qualname__r   r   r   r   r   r   r   r   r   r   r  rS   rS   rS   rU   r      s   

r   c               @   s  e Zd Zejje  ddZdd Zdd Z	edd Z
d	d
 Zdd Zejje  dddd Zejje  dddd Zejje  ddejje  dddd Zejje  ddejje  dddd Zdd Zdd Zdd Zdd Zd d! Zd"d# Zd$d% Zd&d' Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Z d2d3 Z!ee"j#e$d4d5d6 Z%ed7d8 Z&ed9d: Z'ed;d< Z(d=d> Z)d?d@ Z*dAdB Z+dCdD Z,dEdF Z-dGdH Z.dIdJ Z/dKdL Z0dMdN Z1dOdP Z2dQdR Z3dSdT Z4dUdV Z5dWdX Z6dYdZ Z7d[d\ Z8d]d^ Z9d_d` Z:dadb Z;dcdd Z<dedf Z=dgdh Z>didj Z?dkdl Z@dmdn ZAdodp ZBdqdr ZCdsdt ZDdudv ZEejje  dddwdx ZFejjeGjH dyddzd{ ZIejjeGjH d|dd}d~ ZJdd ZKdd ZLdd ZMdd ZNdd ZOdd ZPdd ZQdd ZRdd ZSdd ZTdd ZUdd ZVdd ZWdd ZXdd ZYdd ZZdd Z[dd Z\dd Z]dd Z^dd Z_e"j#e$d4dd Z`ejjdeajbkdddd Zcdd Zddd Zedd Zfdd Zgdd Zhdd Zidd Zjdd Zkdd Zle"j#e$d4dd ZmejjnddĄ ZoddƄ ZpddȄ Zqddʄ Zrdd̄ Zsdd΄ ZtddЄ Zudd҄ ZvddԄ Zwddք Zxdd؄ Zyddڄ Zzdd܄ Z{ddބ Z|dd Z}dd Z~dd Zdd Zdd Zdd Zdd Zdd Zdd Zdd Zdd Zdd ZdddZdd Zdd Zdd Zd d Zdd Zdd Zdd Zdd	 Zd
d Zdd Zdd Zdd Zdd Zdd Ze"j#e$d4dd Ze"j#e$d4dd Ze"j#e$d4dd Ze"j#e$d4dd Zdd Zd d! Zd"d# Zd$d% Zd&d' Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Zd2d3 Zd4d5 Zd6d7 Zd8d9 Zd:d; Zejje  ddd<d= Zd>d? Zd@dA ZdBdC ZdDdE ZdFdG ZdHdI ZdJdK ZdLdM ZdNdO ZdPdQ ZdRdS ZdTdU ZdVdW ZdXdY ZdZd[ Zd\d] Zd^d_ Zd`da Zdbdc Zddde Zdfdg Zejje  dhdZejjejĐdi djdZedkdl ZeeŐdmdn Zedodp Zeedqdr Zeedsdt Zʐdudv ZejjeGjH d|ddwdx ZejjeGjH d|ddydz Z͐d{d| Zΐd}d~ ZϐdS (  ArchiverTestCasezhardlinks not supported)r   c             C   s   t | j| j}t|S )N)r-   _extract_repository_idr   r&   )r   repository_idrS   rS   rU   r&     s    z!ArchiverTestCase.get_security_dirc       
      C   s  | j  }| jdddd| jdd}| jd| | jd| | jd	d
| jd d | jd	d
d| jd d}| jd| | jd| td | jd| jd  W d Q R X | jdd| j}| jd| | jd| ddddddddg}t r|jd t r|jd t r|jd  |s.|j	d |j	d t
rR|j	d tj	tjjdd! | jdd| jd }x|D ]}| j|| qlW | jdd" | jd#| jd }t
rd$nd%}| jd&| | tj| j | jd#| jd }d'd( }	| j|	||	| d S ))Nr   z--encryption=repokeyz--show-versionz	--show-rcT)rL   zborgbackup versionz%terminating with success status, rc 0r   z--exclude-nodumpz::testrN   z--statsz::test.2zArchive name: test.2zThis archive: r^   r   rl   z--shorttestztest.2z
input/bdevz
input/cdevz
input/dir2zinput/dir2/file2zinput/emptyzinput/file1zinput/flagfilezinput/fifo1zinput/link1zinput/hardlinkr   zoutput/inputinfo      zNumber of files: %dc             S   sR   ddddddg}g }x2| j  D ]&}x |D ]}|j|r(|j| q(W qW dj|S )NzName:zFingerprint:zNumber of files:zThis archive:zAll archives:zChunk index:
)r   
startswithappendr   )r^   prefixesresultr   r   rS   rS   rU   filter  s    

z9ArchiverTestCase.test_basic_functionality.<locals>.filter)r  r   r   	assert_inr?   rD   r  rB   rC   r   r<   rb   r   r   assert_dirs_equalru   r   r   r   )
r   r  r^   Zlist_outputexpectedr   Zinfo_outputZ
item_countZinfo_output2r  rS   rS   rU   test_basic_functionality  sX    







z)ArchiverTestCase.test_basic_functionalityc             C   s   t jj| jd}t jj| jd}t j| t j| t jj|d}t jj|d}| j|dd t j|| | jdd| j | jd| jd	 d
d
 | jdd| jd	 }dd |j	dD }t
|d
ddddgkstd S )Nabr   s   123456)r   r   z--encryption=noner   z::testrN   rl   z--json-linesc             S   s   g | ]}|rt j|d  qS )r   )jsonloads)r   r   rS   rS   rU   
<listcomp>  s    z?ArchiverTestCase.test_create_duplicate_root.<locals>.<listcomp>r  zinput/azinput/a/hardlinkzinput/bzinput/b/hardlink)rb   r   r   r   r   r   r   r   r   splitsortedr   )r   path_apath_bhl_ahl_barchive_listpathsrS   rS   rU   test_create_duplicate_root  s    

z+ArchiverTestCase.test_create_duplicate_rootc             C   st   t jj| jdd}t jj|d}| j| }tjtj | j	dd| W d Q R X | j	ddd| t jj
|sptd S )NZparent1Zparent2r   r   z--encryption=nonez--make-parent-dirs)rb   r   r   r   r   pytestraisesr:   ZParentPathDoesNotExistr   r   r   )r   parent_pathr   r   rS   rS   rU   test_init_parent_dirs  s    
z&ArchiverTestCase.test_init_parent_dirsc             C   s   | j dd| j y*tjtjtj}|jtjj| j	d W nP t
k
r } z4|jtjkrdtjd n|jtjkrztjd W Y d d }~X nX | j d| jd d |j  td	* | j d
| jd  tjjd stW d Q R X d S )Nr   z--encryption=repokeyzunix-socketz&unix sockets disabled or not supportedz(permission denied to create unix socketsr   z::testrN   r^   r   zinput/unix-socket)r   r   socketZAF_UNIXZSOCK_STREAMZbindrb   r   r   r   r  r   ZEPERMr)  skipEACCEScloser?   r   r   )r   Zsockr   rS   rS   rU   test_unix_socket  s    
z!ArchiverTestCase.test_unix_socketzsymlinks not supportedc             C   sh   | j   | jdd| j | jd| jd d td* | jd| jd  tjdd	ksZtW d Q R X d S )
Nr   z--encryption=repokeyr   z::testrN   r^   r   zinput/link1r   )r  r   r   r?   rb   readlinkr   )r   rS   rS   rU   test_symlink_extract  s    
z%ArchiverTestCase.test_symlink_extractz4cannot properly setup and execute test without utimec             C   s   dd }| j   d\}}|d}tjd||f | jdd| j | jd| jd	 d
 td | jd| jd	  W d Q R X tjd}tjd}|j|j  ko|d kn  st|r|j	|j	  ko|d kn  stn|j	|d kstd S )Nc             S   sn   t j| j}y(tt j| t}|j  W d Q R X W n tk
rH   dS X t j| j}ttk}|oh||kS d S )NF)rb   r   st_atime_nsr   r   readr  r   )	some_fileatime_beforefileatime_afternoatime_usedrS   rS   rU   has_noatime  s    z0ArchiverTestCase.test_atime.<locals>.has_noatime[8zinput/file1r   z--encryption=repokeyr   z::testrN   r^   r   zoutput/input/file1g    eA)r<  r=  )
r  rb   utimer   r   r?   r   st_mtime_nsr   r4  )r   r;  atimemtimehave_noatimestistorS   rS   rU   
test_atime  s    


$&zArchiverTestCase.test_atimez8cannot properly setup and execute test without birthtimec             C   s   | j   d\}}}tjd||f tjd||f | jdd| j | jd| jd d	 td
 | jd| jd  W d Q R X tjd}tjd}t|jd t|jd   ko|d kn  st	|j
|j
  ko|d kn  st	d S )N k8Cm8 n8zinput/file1r   z--encryption=repokeyr   z::testrN   r^   r   zoutput/input/file1g    eA)rF  rG  rH  )r  rb   r>  r   r   r?   r   intst_birthtimer   r?  )r   	birthtimerA  r@  rC  rD  rS   rS   rU   test_birthtime  s    



4zArchiverTestCase.test_birthtimec             C   s   | j   d\}}}tjd||f tjd||f | jdd| j | jdd| jd	 d
 td | jd| jd	  W d Q R X tjd}tjd}t|jd |d kst	t|jd |d kst	|j
|j
  ko|d kn  st	d S )N k8Cm8 n8zinput/file1r   z--encryption=repokeyr   z--nobirthtimez::testrN   r^   r   zoutput/input/file1g    eA)rM  rN  rO  )r  rb   r>  r   r   r?   r   rI  rJ  r   r?  )r   rK  rA  r@  rC  rD  rS   rS   rU   test_nobirthtime.  s    



z!ArchiverTestCase.test_nobirthtimec          	   C   s   t | j}|jS Q R X d S )N)r:   r   id)r   r   r   rS   rS   rU   r  ?  s    z'ArchiverTestCase._extract_repository_idc             C   sv   t d d}|jtjj|d |jddt| ttjj|dd}|j| W d Q R X t	| j
}|jS Q R X d S )N)interpolationconfigr   rQ  w)r   r5  rb   r   r   setr-   r   r   r:   r   rQ  )r   r   rQ  rS  r   r   rS   rS   rU   _set_repository_idC  s    
z#ArchiverTestCase._set_repository_idc       	      C   sn  dd }t jj| jd}d}ddt>  }|t| | }t|d:}|j|d |j| |j|d |j	 }|j
| W d Q R X ||||}|rj| jdd	| j | jd
| jd d t| j | jdd| jd  W d Q R X | jdd t jj| jdd}t|dH}| j|j|d|  | j|jt|| | j|j|d|  W d Q R X ||||sjtd S )Nc             S   s   t j| }|j|kstd}|r>t|dr>|jd |jkr>d}|rtt drtt drt| dP}y0|jdt jdkrzd}|jdt j	|krd}W n t
k
r   d}Y nX W d Q R X |S )	NT	st_blocksi   F	SEEK_HOLE	SEEK_DATArbr   )rb   r   st_sizer   hasattrrW  r   seekrX  rY  r   )r   
total_size	hole_sizestsparser   rS   rS   rU   	is_sparseM  s    
z4ArchiverTestCase.test_sparse_file.<locals>.is_sparsera  s   foobarr  r;   r   r   z--encryption=repokeyr   z::testrN   r   z--sparsezoutput/inputrZ      )rb   r   r   r   ZCHUNK_MAX_EXPr   r   r]  r   telltruncater   r   r?   r   r  r   r5  r   )	r   rb  r   r   r_  r^  r   posZsparse_supportrS   rS   rU   test_sparse_fileL  s0    
 z!ArchiverTestCase.test_sparse_filec             C   s   dddg}x2|D ]*}t jj| j|}t|d W d Q R X qW | jdd| j | jd| jd d	 xZ|D ]R}td
$ | jd| jd t jjd	| W d Q R X t jjt jjd
d	|sht	qhW d S )NZnormalzwith some blanksz(with_parens)r   r   z--encryption=repokeyr   z::testrN   r^   r   )
rb   r   r   r   r   r   r   r?   r   r   )r   	filenamesr   rS   rS   rU   test_unusual_filenames  s    



(z'ArchiverTestCase.test_unusual_filenamesc             C   s   | j   dtjd< | jdd| j | j| j}| jd| jd d tj| j | jdd| j | j	| j| | j
|| j| j | jr| jd| jd	 dtd
 n,tjtj | jd| jd	 d W d Q R X d S )N
passphraser   r   z--encryption=repokeyr   z::testrN   z--encryption=nonez::test.2)rj   )r  rb   r   r   r   r  r   ru   r   rV  r   r   r,   r)  r*  r   EncryptionMethodMismatch)r   r  rS   rS   rU   test_repository_swap_detection  s    
z/ArchiverTestCase.test_repository_swap_detectionc             C   s   | j   | jdd| jd  dtjd< | jdd| jd  | jd| jd	 d
 tj| jd  tj| jd | jd  | j	r| jd| jd d
t
d n,tjtj | jd| jd d
 W d Q R X d S )Nr   z--encryption=none_unencryptedrj  r   z--encryption=repokey
_encryptedr   z_encrypted::testrN   z_encrypted::test.2)rj   )r  r   r   rb   r   ru   r   r   renamer   r,   r)  r*  r   RepositoryAccessAborted)r   rS   rS   rU   test_repository_swap_detection2  s    
z0ArchiverTestCase.test_repository_swap_detection2c             C   s   | j   dtjd< | jdd| j | j| j}| jd| jd d tj| j | jdd| j | j	| j| | j
|| j| j | jd	d
| j | jr| jd| jd dtd n,tjtj | jd| jd d W d Q R X d S )Nrj  r   r   z--encryption=repokeyr   z::testrN   z--encryption=nonedeletez--cache-onlyz::test.2)rj   )r  rb   r   r   r   r  r   ru   r   rV  r   r   r,   r)  r*  r   rk  )r   r  rS   rS   rU   'test_repository_swap_detection_no_cache  s    
z8ArchiverTestCase.test_repository_swap_detection_no_cachec             C   s   | j   | jdd| jd  dtjd< | jdd| jd  | jd| jd	 d
 | jdd| jd  | jdd| jd  tj| jd  tj| jd | jd  | j	r| jd| jd d
t
d n,tjtj | jd| jd d
 W d Q R X d S )Nr   z--encryption=nonerm  rj  r   z--encryption=repokeyrn  r   z_encrypted::testrN   rr  z--cache-onlyz_encrypted::test.2)rj   )r  r   r   rb   r   ru   r   r   ro  r   r,   r)  r*  r   rp  )r   rS   rS   rU   (test_repository_swap_detection2_no_cache  s    
z9ArchiverTestCase.test_repository_swap_detection2_no_cachec             C   s   | j   | jdd| j | jd| jd d tj| j tdd4 | jdd| j | jdd	| j tj| j  W d Q R X td dR | jr| jd| jd
 dt	d n,t
jtj | jd| jd
 d W d Q R X W d Q R X d S )Nr   z--encryption=repokeyr   z::testrN   rQ   )r   rr  z--cache-onlyz::test.2)rj   )r  r   r   ru   r   r   r@   r&   r   r,   r)  r*  r   CacheInitAbortedError)r   rS   rS   rU   7test_repository_swap_detection_repokey_blank_passphrase  s    zHArchiverTestCase.test_repository_swap_detection_repokey_blank_passphrasec             C   s  | j dd| j | j }tj| j| jd  tdd | j d| jd  W d Q R X ttjj	|d(}|j
 }|t| jd j kstW d Q R X | j d| jd  tj| j | j d| jd  tj| | j d| jd  x&d
D ]}tjjtjj	||stqW d S )Nr   z--encryption=repokeyZ_newyes)Z BORG_RELOCATED_REPO_ACCESS_IS_OKr  locationkey-typemanifest-timestamp)rx  ry  rz  )r   r   r&   rb   ro  r   r@   r   r   r   r5  r%   Zcanonical_pathr   ru   r   r   r   )r   Zsecurity_dirr   rx  r8  rS   rS   rU   test_repository_move  s    $

z%ArchiverTestCase.test_repository_movec          
   C   sP   | j dd| j ttjj| j dd}|jd W d Q R X | j d| j d S )Nr   z--encryption=repokeyrx  rT  zsomething outdatedr  )r   r   r   rb   r   r   r&   r   )r   r   rS   rS   rU   test_security_dir_compat  s    z)ArchiverTestCase.test_security_dir_compatc             C   s   | j dd| j | j d| j tj| j | j d| j tj| j tj| j  | jrl| j d| jtd n&tj	t
j | j d| j W d Q R X tdd | j d| j W d Q R X d S )Nr   z--encryption=noner  )rj   rw  )Z*BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK)r   r   ru   r   r   r&   r   r,   r)  r*  r   ru  r@   )r   rS   rS   rU   test_unknown_unencrypted  s    z)ArchiverTestCase.test_unknown_unencryptedc          !   C   s   | j dd| j | jd | j d| jd d td | j d| jd d	d
 tjjd s`t| jd | j d| jd d	d W d Q R X | jd | j d| jd d	d W d Q R X | jd | j d| jd d	d W d Q R X W d Q R X d S )Nr   z--encryption=repokeyzdir/filer   z::testrN   r^   r   z--strip-components3r8  21zinput/dir/file0)	r   r   r   r?   rb   r   r   r   assert_creates_file)r   rS   rS   rU   test_strip_components  s    

  z&ArchiverTestCase.test_strip_componentsc             C   s   t jt jj| jd t jt jj| jd | jddd t jt jj| jdt jj| jd t jt jj| jdt jj| jd t jt jj| jdt jj| jd | jd	 t jt jj| jd	t jj| jd
 | jdd| j | jd| jd d d S )NZdir1zdir1/subdirsources   123456)r   Zabbazdir1/hardlinkzdir1/subdir/hardlinkzdir1/source2z	dir1/aaaar   z--encryption=repokeyr   z::testrN   )	rb   r   r   r   r   r   r   r   r   )r   rS   rS   rU   _extract_hardlinks_setup  s    
z)ArchiverTestCase._extract_hardlinks_setupzllfuse not installedc          -   C   s*  | j   tjj| jd}tjdkr,ddg}nddg}| j| jd |df|  t	|l tj
djd	ksnttj
d
jd	ksttd
dj dksttj
djd	ksttj
djd	kstW d Q R X W d Q R X | j| jd |df|  t	|v tj
djd	ksttj
djd	ks&ttddj dks>ttj
djd	ksTttj
djd	ksjtW d Q R X W d Q R X | j| jd |f|  t	|v tj
djdksttj
djdksttj
djdksttj
djdksttddj dkstW d Q R X W d Q R X d S )N
mountpointdarwinz-oz$ignore_permissions,defer_permissionsZignore_permissionsz::testz--strip-components=2r   r   zsubdir/hardlinkrZ  s   123456aaaasource2z
input/dir1zinput/dir1/hardlinkzinput/dir1/subdir/hardlinkzinput/dir1/aaaazinput/dir1/source2zinput/sourcer  z
input/abba)r  rb   r   r   r   rW   r   
fuse_mountr   r?   r   st_nlinkr   r   r5  )r   r  Zignore_permsrS   rS   rU   test_fuse_mount_hardlinks1  s4    


(
*
z*ArchiverTestCase.test_fuse_mount_hardlinksc             C   s   | j   td~ | jd| jd  tjdjdks8ttjdjdksLttjdjdks`ttjdjdkstttdd	j	 d
kstW d Q R X d S )Nr^   r   z::testzinput/sourcer  z
input/abbazinput/dir1/hardlinkzinput/dir1/subdir/hardlinkrZ  s   123456)
r  r?   r   r   rb   r   r  r   r   r5  )r   rS   rS   rU   test_extract_hardlinks1S  s    
z(ArchiverTestCase.test_extract_hardlinks1c             C   s.  | j   td | jd| jd dd tjdjdks<ttjdjdksPttdd	j	 d
ksfttjdjdkszttjdjdkstW d Q R X td | jd| jd d tjdjdksttjdjdksttdd	j	 d
ksttjdjdks
ttjdjdks tW d Q R X d S )Nr^   r   z::testz--strip-componentsr  r   r   zsubdir/hardlinkrZ  s   123456r  r  z
input/dir1zinput/dir1/hardlinkzinput/dir1/subdir/hardlinkzinput/dir1/aaaazinput/dir1/source2)
r  r?   r   r   rb   r   r  r   r   r5  )r   rS   rS   rU   test_extract_hardlinks2^  s    

z(ArchiverTestCase.test_extract_hardlinks2c             C   s   t jj| jd}t jj| jd}t j| t j| t jj|d}t jj|d}| j|dd t j|| | jdd| j | jd| jd	 d
d
 t	d@ | jd| jd	  t j
djdkstt j
djdkstW d Q R X d S )Nr  r  r   s   123456)r   r   z--encryption=noner   z::testrN   r^   r   zinput/a/hardlinkr   zinput/b/hardlink)rb   r   r   r   r   r   r   r   r   r?   r   r  r   )r   r"  r#  r$  r%  rS   rS   rU   test_extract_hardlinks_twicep  s    


z-ArchiverTestCase.test_extract_hardlinks_twicec             C   s,  | j dd| j | jddd | jddd | jddd | jd	dd | j d
d| jd d td | j d| jd d W d Q R X | jttjddg td | j dd| jd  W d Q R X | jttjdddg td  | j dd| j | jd  W d Q R X | jttjdddg d S )Nr   z--encryption=repokeyr   i   r   )r   file2file3file4r   z--exclude=input/file4z::testrN   r^   r   zinput/file1zoutput/inputz--exclude=input/file2z--exclude-from=i @ i @ i @ i @ )	r   r   r   r?   r   r!  rb   listdirr   )r   rS   rS   rU   test_extract_include_exclude  s    


$z-ArchiverTestCase.test_extract_include_excludec          #   C   s  | j dd| j | jddd | jddd | jddd | jd	dd | jd
dd | j dd| jd d td | j d| jd  W d Q R X | jttjddddd
g tj	d td | j dd| jd  W d Q R X | jttjdddg tj	d td | j ddd| jd  W d Q R X | jttjddd
g tj	d td$ | j dd| j
 dd| jd  W d Q R X | jttjddg d S )Nr   z--encryption=repokeyr   i   r   )r   r  r  r  file333r   z--exclude=re:input/file4$z::testrN   r^   r   zoutput/inputz--exclude=re:file3+z--exclude=input/file2z--exclude=re:file[01]z--exclude-from=z--exclude=re:file1z--exclude=re:file(\d)\1\1$i @ i @ i @ i @ i @ )r   r   r   r?   r   r!  rb   r  ru   r   r   )r   rS   rS   rU   "test_extract_include_exclude_regex  s.    




 

z3ArchiverTestCase.test_extract_include_exclude_regexc          /   C   s  | j dd| j | jddd | jddd | jddd | jd	dd | jd
dd | jdd d t| jd}|jd |jd W d Q R X | j dd| j | jd d td | j d| jd  W d Q R X | jtt	j
ddddd
g tjd t| jd}|jd W d Q R X td  | j dd| j | jd  W d Q R X | jtt	j
dddg tjd t| jd.}|jd |jd |jd |jd W d Q R X td  | j dd| j | jd  W d Q R X | jtt	j
ddg d S )!Nr   z--encryption=repokeyr   i   r   )r   r  r  r  r  zaa:somethingr   s   re:input/file4$
s   fm:*aa:*thing
r   z--exclude-from=z::testrN   r^   r   zoutput/inputs
   re:file3+
s   re:file(\d)\1\1$
s   fm:nothingwillmatchthis
s   */file1
s
   re:file2$
i @ i @ i @ i @ i @ i @ )r   r   r   r   r   r   r?   r   r!  rb   r  ru   r   )r   r   rS   rS   rU   ,test_extract_include_exclude_regex_from_file  s:    



$




$z=ArchiverTestCase.test_extract_include_exclude_regex_from_filec          "   C   s  | j dd| j | jddd | jddd | jddd | jd	dd | jd
dd | j d| jd d td | j d| jd d W d Q R X | jttjddddd
d	g tj	d td | j dd| jd d W d Q R X | jttjdddg tj	d td | j dd| jd d W d Q R X | jttjddg tj	d td | j d| jd ddd W d Q R X | jttjdddd
g d S )Nr   z--encryption=repokeyr   i   r   )r   r  r  r  r  r   z::testrN   r^   r   zre:.*zoutput/inputz--exclude=re:file[34]$z
re:file\d$z--exclude=input/file1zre:file[12]$zfm:input/file1zfm:*file33*zinput/file2i @ i @ i @ i @ i @ )
r   r   r   r?   r   r!  rb   r  ru   r   )r   rS   rS   rU   test_extract_with_pattern  s,    
 

 

 

"z*ArchiverTestCase.test_extract_with_patternc          "   C   s$  | j dd| j | jddd | j d| jd d	 td
 | j d| jd }W d Q R X | jd| tjd td
 | j dd| jd }W d Q R X | jd| tjd td
 | j dd| jd }W d Q R X | jd| tjd td
 | j ddd| jd }W d Q R X | jd| d S )Nr   z--encryption=repokeyr8  i   r   )r   r   z::testrN   r^   r   z
input/filezoutput/inputz--infoz--listi @ )r   r   r   r?   assert_not_inru   r   r  )r   r^   rS   rS   rU   test_extract_list_output  s$    






 z)ArchiverTestCase.test_extract_list_outputc             C   sj   | j dd| j | jddd | j d| jd d	 td
& | j d| jd d}d|ks\tW d Q R X d S )Nr   z--encryption=repokeyr8  i   r   )r   r   z::testrN   r^   r   z
--progresszExtracting:i @ )r   r   r   r?   r   )r   r^   rS   rS   rU   test_extract_progress  s    
z&ArchiverTestCase.test_extract_progressc             C   s   | j dd| j | jddd | jdt td d	 | jd
t dd	 tjd t rltjdt dt  n| jdt td d	 d S )Nr   z--encryption=repokeyr   i   r   )r   z	cache1/%ss    extra stuff)r   z	cache2/%ss   invalid signaturezinput/cache3zinput/cache1/%szinput/cache3/%sz	cache3/%si @ )	r   r   r   CACHE_TAG_NAMECACHE_TAG_CONTENTSrb   r   rC   r   )r   rS   rS   rU   _create_test_caches$  s    



z$ArchiverTestCase._create_test_cachesc             C   s   | j dd| j d}| j d| jd d|d tj| j dd	| jd }|d
 dksVt|d dksft|d t|kszt|d dkst| j dd| jd dd}||kstd S )Nr   z--encryption=repokeys    foo

bar
   
r   z::test-)rN   rl   z--json-linesuidr   gidr   r   rd   r   z--stdoutT)rO   )r   r   r  r  r   r   )r   Z
input_dataitemZextracted_datarS   rS   rU   test_create_stdin2  s    z"ArchiverTestCase.test_create_stdinc             C   s*   | j dd| j | j d| jd dd dS )ztest create without a rootr   z--encryption=repokeyr   z::testr   )rj   N)r   r   )r   rS   rS   rU   test_create_without_root>  s    z)ArchiverTestCase.test_create_without_rootc             C   s`   | j dd| j | jddd | jddd | j dd	d
d| jd }| jd| | jd| dS )z$test create with only a root patternr   z--encryption=repokeyr   i   r   )r   r  r   z-vz--listz--pattern=R inputz::testzA input/file1zA input/file2Ni @ i @ )r   r   r   r  )r   r^   rS   rS   rU   test_create_pattern_rootC  s    z)ArchiverTestCase.test_create_pattern_rootc             C   s~   | j dd| j | jddd | jddd | jddd | j d	d
ddd| jd d}| jd| | jd| | jd| dS )z test file patterns during creater   z--encryption=repokeyr   i   r   )r   r  file_importantr   z-vz--listz--pattern=+input/file_importantz--pattern=-input/file*z::testrN   zA input/file_importantzx input/file1zx input/file2Ni @ i @ i @ )r   r   r   r  )r   r^   rS   rS   rU   test_create_patternL  s    
z$ArchiverTestCase.test_create_patternc             C   s   | j dd| j | jddd | jddd | jddd | jd	dd | j d
dddd| j | jd d}| jd| | jd| | jd| | jd| dS )z test file patterns during creater   z--encryption=repokeyr   i   r   )r   r  Z	otherfiler  r   z-vz--listz--pattern=-input/otherfilez--patterns-from=z::testrN   zA input/file_importantzx input/file1zx input/file2zx input/otherfileNi @ i @ i @ i @ )r   r   r   r   r  )r   r^   rS   rS   rU   test_create_pattern_fileY  s    

z)ArchiverTestCase.test_create_pattern_filec             C   s   t jj| jd| _t| jd}|jd W dQ R X | jdd| j | j	ddd
 | j	ddd
 | j	ddd
 | jdddd| j | jd d}| j
d| | j
d| | j
d| dS )z?test when patterns exclude a parent folder, but include a child	patterns2r   s   + input/x/b
- input/x*
Nr   z--encryption=repokeyz	x/a/foo_ai   r   )r   z	x/b/foo_bzy/foo_yr   z-vz--listz--patterns-from=z::testrN   zx input/x/a/foo_azA input/x/b/foo_bzA input/y/foo_yi @ i @ i @ )rb   r   r   r   patterns_file_path2r   r   r   r   r   r  )r   r   r^   rS   rS   rU   .test_create_pattern_exclude_folder_but_recurseh  s    
z?ArchiverTestCase.test_create_pattern_exclude_folder_but_recursec             C   s   t jj| jd| _t| jd}|jd W dQ R X | jdd| j | j	ddd
 | j	ddd
 | j	ddd
 | jdddd| j | jd d}| j
d| | j
d| | jd| dS )zCtest when patterns exclude a parent folder and, but include a childr  r   s   + input/x/b
! input/x*
Nr   z--encryption=repokeyz	x/a/foo_ai   r   )r   z	x/b/foo_bzy/foo_yr   z-vz--listz--patterns-from=z::testrN   zinput/x/a/foo_az	input/x/azA input/y/foo_yi @ i @ i @ )rb   r   r   r   r  r   r   r   r   r   r  r  )r   r   r^   rS   rS   rU   -test_create_pattern_exclude_folder_no_recursey  s    
z>ArchiverTestCase.test_create_pattern_exclude_folder_no_recursec             C   s   t jj| jd| _t| jd}|jd W dQ R X | jdd| j | j	ddd
 | j	ddd
 t
d" | jdd| j | jd d W dQ R X | jddd| jd }|j }| jd| | jd| |jd|jdk st|jd|jdk stdS )zetest that intermediate folders appear first when patterns exclude a parent folder but include a childr  r   s#   + input/x/a
+ input/x/b
- input/x*
Nr   z--encryption=repokeyz	x/a/foo_ai   r   )r   z	x/b/foo_brN   r   z--patterns-from=z::test.rl   z--formatz{type} {path}{NL}zd x/azd x/bz- x/a/foo_az- x/b/foo_bi @ i @ )rb   r   r   r   r  r   r   r   r   r   r?   r   r  indexr   )r   r   r   Zout_listrS   rS   rU   .test_create_pattern_intermediate_folders_first  s    
z?ArchiverTestCase.test_create_pattern_intermediate_folders_firstc             C   s   | j   | jdd| j | jdd| j tj| jdd| jd dd	d
}tj| jd| jd d	d
}|d d }|d d }||kst| jdd| j | jdd| jd d | jd| j | jd| j d S )Nr   z--encryption=repokeyrr  z--cache-onlyr   z--no-cache-syncz::testrN   z--jsonz--errorr  r   statsz::test2r   )r  r   r   r  r  r   )r   create_jsonZ	info_jsonZcreate_statsZ
info_statsrS   rS   rU   test_create_no_cache_sync  s    
z*ArchiverTestCase.test_create_no_cache_syncc             C   s   | j dd| j | jddd | jddd | jddd | j d	| jd
 d td | j ddd| jd
  W d Q R X | jttjddg d S )Nr   z--encryption=repokeyr   i   r   )r   r  r  r   z::testrN   r^   r   z--pattern=+input/file_importantz--pattern=-input/file*zoutput/inputi @ i @ i @ )r   r   r   r?   r   r!  rb   r  )r   rS   rS   rU   test_extract_pattern_opt  s    
z)ArchiverTestCase.test_extract_pattern_optc             C   s\   t d | jd| jd  W d Q R X | jttjdddg | jttjdtg d S )Nr^   r   z::testzoutput/inputZcache2r   zoutput/input/cache2)r?   r   r   r   r!  rb   r  r  )r   rS   rS   rU   _assert_test_caches  s    
z$ArchiverTestCase._assert_test_cachesc             C   s*   | j   | jdd| jd d | j  d S )Nr   z--exclude-cachesz::testrN   )r  r   r   r  )r   rS   rS   rU   test_exclude_caches  s    z$ArchiverTestCase.test_exclude_cachesc             C   s<   | j   | jd| jd d | jdd| jd  | j  d S )Nr   z::testrN   recreatez--exclude-caches)r  r   r   r  )r   rS   rS   rU   test_recreate_exclude_caches  s    z-ArchiverTestCase.test_recreate_exclude_cachesc             C   sD   | j dd| j | jdd
d | jd | jd | jd	dd d S )Nr   z--encryption=repokeyr   i   r   )r   ztagged1/.NOBACKUPztagged2/00-NOBACKUPztagged3/.NOBACKUP/file2i @ )r   r   r   )r   rS   rS   rU   _create_test_tagged  s
    

z$ArchiverTestCase._create_test_taggedc             C   sB   t d | jd| jd  W d Q R X | jttjddg d S )Nr^   r   z::testzoutput/inputr   )r?   r   r   r   r!  rb   r  )r   rS   rS   rU   _assert_test_tagged  s    
z$ArchiverTestCase._assert_test_taggedc             C   s0   | j   | jddddd| jd d | j  d S )Nr   z--exclude-if-presentz	.NOBACKUPz00-NOBACKUPz::testrN   )r  r   r   r  )r   rS   rS   rU   test_exclude_tagged  s    z$ArchiverTestCase.test_exclude_taggedc             C   sB   | j   | jd| jd d | jddddd| jd  | j  d S )Nr   z::testrN   r  z--exclude-if-presentz	.NOBACKUPz00-NOBACKUP)r  r   r   r  )r   rS   rS   rU   test_recreate_exclude_tagged  s
    z-ArchiverTestCase.test_recreate_exclude_taggedc             C   s   | j dd| j | jddd | jd | jddd | jddd | jd	dd | jd
t td d | jddd | jd | jddd | jdt td d | jddd d S )Nr   z--encryption=repokeyfile0i   )r   ztagged1/.NOBACKUP1ztagged1/file1ztagged2/.NOBACKUP2/subfile1ztagged2/file2z
tagged3/%ss    extra stuff)r   ztagged3/file3ztaggedall/.NOBACKUP1ztaggedall/.NOBACKUP2/subfile1ztaggedall/%sztaggedall/file4)r   r   r   r  r  )r   rS   rS   rU   _create_test_keep_tagged  s    



z)ArchiverTestCase._create_test_keep_taggedc             C   s   t d | jd| jd  W d Q R X | jttjdddddd	g | jtjd
dg | jtjddg | jtjdtg | jttjdddtg d S )Nr^   r   z::testzoutput/inputr  Ztagged1Ztagged2Ztagged3Z	taggedallzoutput/input/tagged1z
.NOBACKUP1zoutput/input/tagged2z
.NOBACKUP2zoutput/input/tagged3zoutput/input/taggedall)r?   r   r   r   r!  rb   r  r  )r   rS   rS   rU   _assert_test_keep_tagged  s    
 z)ArchiverTestCase._assert_test_keep_taggedc             C   s8   | j dd| j | j ddd| jd t}| jd| d S )Nr   z--encryption=repokeyr   z--exclude-cachesz--keep-tag-filesz::testz&--keep-tag-files" has been deprecated.)r   r   r   r  )r   output_warnrS   rS   rU   $test_exclude_keep_tagged_deprecation  s    z5ArchiverTestCase.test_exclude_keep_tagged_deprecationc          
   C   s4   | j   | jddddddd| jd d	 | j  d S )	Nr   z--exclude-if-presentz
.NOBACKUP1z
.NOBACKUP2z--exclude-cachesz--keep-exclude-tagsz::testrN   )r  r   r   r  )r   rS   rS   rU   test_exclude_keep_tagged  s    z)ArchiverTestCase.test_exclude_keep_taggedc          
   C   sF   | j   | jd| jd d | jddddddd	| jd  | j  d S )
Nr   z::testrN   r  z--exclude-if-presentz
.NOBACKUP1z
.NOBACKUP2z--exclude-cachesz--keep-exclude-tags)r  r   r   r  )r   rS   rS   rU   !test_recreate_exclude_keep_tagged  s
    z2ArchiverTestCase.test_recreate_exclude_keep_taggedc             C   s   | j dd| j | jdtd tjtjj| jd tj	tjj| jdtjj| jdt
 | j d| jd d | j d	d
d| jd  d S )Nr   z--encryption=noner   )r   Zsubdirr   z::testrN   r  z--exclude-cachesz--keep-exclude-tags)r   r   r   r  rb   r   r   r   r   r   r  )r   rS   rS   rU   test_recreate_hardlinked_tags  s    z.ArchiverTestCase.test_recreate_hardlinked_tagsz4Linux capabilities test, requires fakeroot >= 1.20.2c                s   t j  fdd}d}| jd tjdd| | jdd| j | jd	| jd
 d tdF tj	t d| | jd| jd
  W d Q R X tj
dd|kstW d Q R X d S )Nc                s"   t j| dd dd  | || d S )Nzsecurity.capabilityF)follow_symlinks)r   r   )r   r  r  )fchownrS   rU   patched_fchown'  s    zBArchiverTestCase.test_extract_capabilities.<locals>.patched_fchowns                     r8  z
input/filezsecurity.capabilityr   z--encryption=repokeyr   z::testrN   r^   r  r   )rb   r  r   r   r   r   r   r?   r   objectgetxattrr   )r   r  ZcapabilitiesrS   )r  rU   test_extract_capabilities!  s    

z*ArchiverTestCase.test_extract_capabilitiesz@xattr not supported on this system or on this version offakerootc          !   C   s`  dd }dd }dd }| j d tjdd	d
 | jd| jd | jd| jd d td tjjd}t	j
td|, | jd| jd td}||d kstW d Q R X tj| t	j
td|, | jd| jd td}||d kstW d Q R X tj| t	j
td|. | jd| jd td}||d ks6tW d Q R X tjj|sRtW d Q R X d S )Nc              _   s   t tjdd S )NE2BIG)r   r   r  )rT   kwargsrS   rS   rU   patched_setxattr_E2BIG9  s    zKArchiverTestCase.test_extract_xattrs_errors.<locals>.patched_setxattr_E2BIGc              _   s   t tjdd S )NENOTSUP)r   r   r  )rT   r  rS   rS   rU   patched_setxattr_ENOTSUP<  s    zMArchiverTestCase.test_extract_xattrs_errors.<locals>.patched_setxattr_ENOTSUPc              _   s   t tjdd S )Nr/  )r   r   r/  )rT   r  rS   rS   rU   patched_setxattr_EACCES?  s    zLArchiverTestCase.test_extract_xattrs_errors.<locals>.patched_setxattr_EACCESr8  z
input/filezuser.attributevaluer   z-enoner   z::testrN   r^   r   r   )rj   zN: when setting extended attribute user.attribute: too big for this filesystem
zZ: when setting extended attribute user.attribute: xattrs not supported on this filesystem
zD: when setting extended attribute user.attribute: Permission denied
)r   r   r   r   r   r?   rb   r   abspathr   r  r+   r   r   isfile)r   r  r  r  Zinput_abspathr   rS   rS   rU   test_extract_xattrs_errors6  s*    



z+ArchiverTestCase.test_extract_xattrs_errorsc             C   st   | j dd| j | jddd td | j d| jd	 d
 W d Q R X | j d| jd	 }| jd| | jd| d S )Nr   z--encryption=repokeyzdir1/dir2/filei   r   )r   zinput/dir1/dir2r   z::testz#../../../input/dir1/../dir1/dir2/..rl   z..z input/dir1/dir2/filei @ )r   r   r   r?   r  r  )r   r^   rS   rS   rU   test_path_normalizationU  s    
z(ArchiverTestCase.test_path_normalizationc          '   C   sT  | j dd| j | jddd | jddd td | j d	d
| jd d W d Q R X td | j d| jd  W d Q R X | jttjddg td | j d	d| jd d W d Q R X td | j d| jd  W d Q R X | jttjddg | j d	d| jd d td | j d| jd  W d Q R X | jttjddg d S )Nr   z--encryption=repokeyr   i   r   )r   r  rN   r   z--exclude=file1z::test1r  r^   r   z--exclude=./file1z::test2z--exclude=input/./file1z::test3zoutput/inputi @ i @ )r   r   r   r?   r   r!  rb   r  )r   rS   rS   rU   test_exclude_normalization^  s"    
 

 

z+ArchiverTestCase.test_exclude_normalizationc             C   s8   | j dd
d | jdd| j | jd| jd d	d	 d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   i @ )r   r   r   )r   rS   rS   rU   test_repeated_filesq  s    z$ArchiverTestCase.test_repeated_filesc             C   s   | j ddd | j ddd | jdd| j | jd| jd	 d
 tjd tjd tjd td | jd| jd	  W d Q R X | jd
d tjd tjd tjd td | jd| jd	 dd W d Q R X d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   zoutput/inputzoutput/input/file1zoutput/input/dir2r^   r   zoutput/input/file1/dirr;   )rj   i @ i @ )r   r   r   rb   r   r?   r  unlink)r   rS   rS   rU   test_overwritev  s    







zArchiverTestCase.test_overwritec          
   C   s:  | j ddd | j ddd | jdd| j | jd| jd	 d
 | jd| jd d
 | jdd| jd	  | jdd| jd  | jd| jd	 d | jdd| jd  | jd| jd d | jdd| jd  | jdd| jd  t| j}tj|tj\}}W d Q R X | jt	|j
d | jd|j
 | jd|j
 d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   z::test.2r   z	--dry-runro  ztest.3ztest.4z::test.3z::test.4r   i @ i @ )r   r   r   r:   r   r'   r   r   r   r   archivesr  )r   r   r   rI   rS   rS   rU   test_rename  s"    zArchiverTestCase.test_renamec             C   s   | j ddd | jdd| j | jd| jd d	 | jd
| j}d|ksLt| jd
| jd }d|ksjt| jd
dd| j}d|kstd S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   r  zAll archives:zArchive name: test
z--firstr  i @ )r   r   r   r   )r   	info_repoinfo_archiverS   rS   rU   	test_info  s    zArchiverTestCase.test_infoc                s  | j dd(d | jdd| j | jd| jd d	 tj| jd
d| j}|d }t|d dksdtd|kspttj|d t	st|d d dkstd|d kst|d }|d  t
dd  j D stt
 fddd)D sttj| jd
d| jd }|d |d kst|d |d ks2t|d }t|d ksLt|d! }|d" d#ksftt|d$ tsztt|d% tstt|d dkstd|ksttj|d& t	sttj|d' t	std S )*Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   r  z--jsonr   rQ  @   last_modified
encryptionmoderepokeykeyfiler   r  c             s   s   | ]}t |tV  qd S )N)rY   rI  )r   orS   rS   rU   r     s    z2ArchiverTestCase.test_info_json.<locals>.<genexpr>c             3   s   | ]}| kV  qd S )NrS   )r   rI   )r  rS   rU   r     s    total_chunkstotal_csizer^  total_unique_chunksunique_csizeunique_sizer  r;   r   r   r  command_linedurationstartendi @ )r  r  r^  r  r  r  )r   r   r   r  r  r   r   r   strptime
ISO_FORMATallvaluesrY   rl   float)r   r  r   r   r  r  r   rS   )r  rU   test_info_json  s6    zArchiverTestCase.test_info_jsonc             C   sd   | j dd| j tj| j ddd| j}|d g ks8ttj| j ddd| j}|d g ks`tdS )	z2See https://github.com/borgbackup/borg/issues/6120r   z--encryption=repokeyr  z--jsonz	--first=1r  z--last=1N)r   r   r  r  r   )r   r  rS   rS   rU   test_info_json_of_empty_archive  s
    z0ArchiverTestCase.test_info_json_of_empty_archivec             C   sx  | j ddd | jdd| j | jd| jd d	 | jdd
d| jd d	 | jdd
d| jd d	 | jdd
d| jd d	 d| jd| jd kstd| jd| jd kst| jd| jd d
d | jd| jd d
d | jd| jd d
d | jd| jd d d| jd| jd ks td| jd| jd ks<td| jd| jd ksXtd| jd| jd ksttd S )Nr   i   r   )r   r   z--encryption=repokeyr   z::test1rN   z	--commentzthis is the commentz::test2z"deleted" commentz::test3zpreserved commentz::test4z
Comment: 
r  zComment: this is the commentr  zadded commentzmodified commentrQ   Z12345zComment: added commentzComment: modified commentzComment: preserved commenti @ )r   r   r   r   )r   rS   rS   rU   test_comment  s     zArchiverTestCase.test_commentc          
   C   sL  | j ddd | j ddd | jdd| j | jd| jd	 d
 | jd| jd d
 | jd| jd d
 | jd| jd d
 | jd| jd d
 | jdd| jd	  | jdd| jd  | jddd| j | jddd| j | jd| jd	  | jdd| jd  | jdd| jd }| jd| t| j}| jt|d W d Q R X d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   z::test.2z::test.3z::another_test.1z::another_test.2r   z	--dry-runrr  z--prefixZanother_z--lastr  z--statszDeleted data:r;   i @ i @ )r   r   r   r  r:   r   r   r   )r   r^   r   rS   rS   rU   test_delete  s$    zArchiverTestCase.test_deletec             C   s   | j ddd | jdd| j | jd| jd d	 | jd| jd
 d	 | jd| jd d	 | jd| jd d | jdd| jd  | jd| jd | jd| j std S )Nr   i   r   )r   r   z--encryption=repokeyr   z::test1rN   z::test2z::test3rr  test2r   z	--dry-runZtest3rl   i @ )r   r   r   r   )r   rS   rS   rU   test_delete_multiple  s    z%ArchiverTestCase.test_delete_multiplec             C   s   | j ddd | j ddd | jdd| j | jd| jd	 d
 | jd| jd d
 dtjd< | jd| jdd tjj| jstdtjd< | jd| j | j	tjj| j d S )Nr   i   r   )r   z
dir2/file2r   z--encryption=repokeyr   z::testrN   z::test.2nor   rr  r   )rj   r   i @ i @ )
r   r   r   rb   r   r   r   r   r   ZassertFalse)r   rS   rS   rU   test_delete_repo  s    

z!ArchiverTestCase.test_delete_repoc             C   s   | j dd| j | jd t| jddh}tj|tj\}}t|||d}x8|j	 D ]$}|j
jdrT|j|jd j P qTW dst|j  W d Q R X | j d	d
| jd }| jd| | j dd| j | j d| j}| jd| d S )Nr   z--encryption=noner  T)r   ztestsuite/archiver.pyr;   Frr  z--forcez::testzdeleted archive was corruptedr   z--repairrl   )r   r   r   r:   r   r'   r   r   r   
iter_itemsr   endswithrr  chunksrQ  r   commitr  r  )r   r   r   rI   r   r  r^   rS   rS   rU   test_delete_force  s     
z"ArchiverTestCase.test_delete_forcec             C   s   | j dd| j | jd t| jddF}tj|tj\}}t|||d}|j	j
d }|j|d |j  W d Q R X | j dd	d	| jd
  | j dd| j | j d| j}| jd| d S )Nr   z--encryption=noner  T)r   r   s%   corrupted items metadata stream chunkrr  z--forcez::testr   z--repairrl   )r   r   r   r:   r   r'   r   r   r   metadataitemsputr  r  )r   r   r   rI   r   rQ  r^   rS   rS   rU   test_delete_double_force"  s    
z)ArchiverTestCase.test_delete_double_forcec             C   s   | j dd| j | jd | j dd| jd  | j dd| j}| jd	| | jd
| ttjtjj	| j
dddddd }ttjj	| j
ddd|d}|jd |jd W d Q R X | j dd| jdd}| jd
| d S )Nr   z--encryption=repokeyr  r   z	--dry-runz::testr   z--show-versionzborgbackup versionzStarting repository checkr   r   r  T)reverser;   zr+br   s   XXXXz--info)rj   )r   r   r   r  r  r!  rb   r  r   r   r   r   r]  r   )r   r^   r   r   rS   rS   rU   test_corrupted_repository0  s    
&
z*ArchiverTestCase.test_corrupted_repositoryc             C   s   | j dd| j | jd | j| j~ | jrD| j dd| jtd nJtjt	t
jf}| j dd| j W d Q R X t|jt
jr|jjdkst| j dd| jd W d Q R X d S )	Nr   z--encryption=repokeyr  r   z--verify-data)rj   r6   z--bypass-lock)r   r   r   	read_onlyr   r   r,   r)  r*  r6   r8   RPCErrorrY   r  exception_classr   )r   excinforS   rS   rU   test_readonly_check?  s    
z$ArchiverTestCase.test_readonly_checkc             C   s   | j dd| j | jd | jd | j| j | jrR| j dd| j dtd nNtjt	t
jf}| j dd| j d W d Q R X t|jt
jr|jjdkst| j dd| j dd	 W d Q R X d S )
Nr   z--encryption=repokeyr  r  diffz%s::a)rj   r6   z--bypass-lock)r   r   r   r  r   r   r,   r)  r*  r6   r8   r  rY   r  r	  r   )r   r
  rS   rS   rU   test_readonly_diffN  s    

z#ArchiverTestCase.test_readonly_diffc             C   s   | j dd| j | jd | j| j | jrH| j dd| j dtd nNtjt	t
jf}| j dd| j d W d Q R X t|jt
jr|jjdkst| j dd| j dd	 W d Q R X d S )
Nr   z--encryption=repokeyr  z
export-tarz%s::testztest.tar)rj   r6   z--bypass-lock)r   r   r   r  r   r   r,   r)  r*  r6   r8   r  rY   r  r	  r   )r   r
  rS   rS   rU   test_readonly_export_tar^  s    
z)ArchiverTestCase.test_readonly_export_tarc             C   s   | j dd| j | jd | j| j | jrF| j dd| j td nLtjt	t
jf}| j dd| j  W d Q R X t|jt
jr|jjdkst| j dd| j d W d Q R X d S )	Nr   z--encryption=repokeyr  r   z%s::test)rj   r6   z--bypass-lock)r   r   r   r  r   r   r,   r)  r*  r6   r8   r  rY   r  r	  r   )r   r
  rS   rS   rU   test_readonly_extractm  s    
z&ArchiverTestCase.test_readonly_extractc             C   s   | j dd| j | jd | j| jx | jrB| j d| jtd nHtjt	t
jf}| j d| j W d Q R X t|jt
jr|jjdkst| j d| jd W d Q R X d S )Nr   z--encryption=repokeyr  r  )rj   r6   z--bypass-lock)r   r   r   r  r   r   r,   r)  r*  r6   r8   r  rY   r  r	  r   )r   r
  rS   rS   rU   test_readonly_info|  s    
z#ArchiverTestCase.test_readonly_infoc             C   s   | j dd| j | jd | j| jx | jrB| j d| jtd nHtjt	t
jf}| j d| j W d Q R X t|jt
jr|jjdkst| j d| jd W d Q R X d S )Nr   z--encryption=repokeyr  rl   )rj   r6   z--bypass-lock)r   r   r   r  r   r   r,   r)  r*  r6   r8   r  rY   r  r	  r   )r   r
  rS   rS   rU   test_readonly_list  s    
z#ArchiverTestCase.test_readonly_listc             C   s   | j dd| j | jd | j| j | jrL| j| jtd W d Q R X nVtj	t
tjf"}| j| jdd W d Q R X W d Q R X t|jtjr|jjdkst| j| jd d W d Q R X W d Q R X d S )	Nr   z--encryption=repokeyr  )rj   F)rL   r6   z--bypass-lock)r   r   r   r  r   r   r  r,   r)  r*  r6   r8   r  rY   r  r	  r   )r   r
  rS   rS   rU   test_readonly_mount  s    
z$ArchiverTestCase.test_readonly_mountBORG_TESTS_IGNORE_MODESzmodes unreliablec             C   sV   | j ddd | jdd| j | jd| jd d	 tj| jj}| jtj|d
 d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   i  i @ )	r   r   r   rb   r   r   st_modeassertEqualS_IMODE)r   r  rS   rS   rU   
test_umask  s
    zArchiverTestCase.test_umaskc          
   C   sd   | j dd| j | j dd| jd d t| j}tj|tj\}}W d Q R X | jt|j	d d S )Nr   z--encryption=repokeyr   z	--dry-runz::testrN   r   )
r   r   r:   r   r'   r   r   r   r   r  )r   r   r   rI   rS   rS   rU   test_create_dry_run  s
    z$ArchiverTestCase.test_create_dry_runc          
   C   sZ   t | jddB}tj|tj\}}|jj ddgii|jd< |j  |j	  W d Q R X d S )NT)r   s	   mandatorys   unknown-features   feature_flags)
r:   r   r'   r   r   r  r   rS  r   r  )r   Z	operationr   r   rI   rS   rS   rU   add_unknown_feature  s
    z$ArchiverTestCase.add_unknown_featurec          	   C   sP   | j r| j|dti n4tjt}| j|  W d Q R X |jjdgfksLtd S )Nrj   zunknown-feature)	r   r   r,   r)  r*  r(   r  rT   r   )r   rT   r
  rS   rS   rU   cmd_raises_unknown_feature  s
    z+ArchiverTestCase.cmd_raises_unknown_featurec             C   s<   t | jdd| j | jtjj | jd| jd dg d S )Nr   z--encryption=repokeyr   z::testrN   )r   r   r   r  r'   	OperationZWRITEr  )r   rS   rS   rU   test_unknown_feature_on_create  s    z/ArchiverTestCase.test_unknown_feature_on_createc             C   sH   | j dd| j | j dd| j | jtjj | jd| jd dg d S )Nr   z--encryption=repokeyrr  z--cache-onlyr   z::testrN   )r   r   r  r'   r  READr  )r   rS   rS   rU   "test_unknown_feature_on_cache_sync  s    z3ArchiverTestCase.test_unknown_feature_on_cache_syncc             C   s6   t | jdd| j | jtjj | jd| jg d S )Nr   z--encryption=repokeyzchange-passphrase)r   r   r   r  r'   r  CHECKr  )r   rS   rS   rU   )test_unknown_feature_on_change_passphrase  s    z:ArchiverTestCase.test_unknown_feature_on_change_passphrasec             C   s   t | jdd| j | jd| jd d | jtjj td | jd| jd g W d Q R X | jd| jg | jd	| jd g d S )
Nr   z--encryption=repokeyr   z::testrN   r^   r   rl   r  )	r   r   r   r  r'   r  r  r?   r  )r   rS   rS   rU   test_unknown_feature_on_read  s    
z-ArchiverTestCase.test_unknown_feature_on_readc             C   sP   t | jdd| j | jd| jd d | jtjj | jd| jd dg d S )Nr   z--encryption=repokeyr   z::testrN   ro  other)r   r   r   r  r'   r  r  r  )r   rS   rS   rU   test_unknown_feature_on_rename  s    z/ArchiverTestCase.test_unknown_feature_on_renamec             C   sn   t | jdd| j | jd| jd d | jtjj | jd| jd g | jdd| jg | jd| j d S )	Nr   z--encryption=repokeyr   z::testrN   rr  prunez--keep-daily=3)r   r   r   r  r'   r  ZDELETEr  )r   rS   rS   rU   test_unknown_feature_on_delete  s    z/ArchiverTestCase.test_unknown_feature_on_deletec             C   sf   | j dd| j | j d| jd d | jtjj tjj| j	d}tj
| | jd| jd |g d S )Nr   z--encryption=repokeyr   z::testrN   r  r   )r   r   r  r'   r  r  rb   r   r   r   r   r  )r   r  rS   rS   rU   test_unknown_feature_on_mount  s    
z.ArchiverTestCase.test_unknown_feature_on_mountc          &      sr  | j rd}nd}t| jdd| j t| jdd^}|rDt| j|_tj	|tj
\}}t|||$}|j  tdg|j_|j  W d Q R X W d Q R X | jr| jd| jd	 d
 nPd tj fdd}tjtd| | jd| jd	 d
 W d Q R X  stt| jddX}|r$t| j|_tj	|tj
\}}t|||}|jjtg ksZtW d Q R X W d Q R X d S )Nzssh://__testsuite__rQ   r   z--encryption=repokeyT)r   zunknown-featurer   z::testrN   Fc                 s   d |   d S )NTrS   )rT   )calledwipe_cache_saferS   rU   wipe_wrapper  s    zNArchiverTestCase.test_unknown_mandatory_feature_in_cache.<locals>.wipe_wrapper
wipe_cache)r   r   r   r   r:   r   r%   Z	_locationr'   r   r   r   	begin_txnrU  Zcache_configZmandatory_featuresr  r   r   r*  r   r  r   )r   path_prefixr   r   rI   r   r)  rS   )r'  r(  rU   'test_unknown_mandatory_feature_in_cache  s4    
z8ArchiverTestCase.test_unknown_mandatory_feature_in_cachec             C   sD   | j ddd | jdd| j | jdd| jd	 d
}| jd| d S )Nr   i   r   )r   r   z--encryption=repokeyr   z
--progressz::test4rN   i @ )r   r   r   r  )r   r^   rS   rS   rU   test_progress_on*  s    z!ArchiverTestCase.test_progress_onc             C   sB   | j ddd | jdd| j | jd| jd d	}| jd
| d S )Nr   i   r   )r   r   z--encryption=repokeyr   z::test5rN   r.  i @ )r   r   r   r  )r   r^   rS   rS   rU   test_progress_off0  s    z"ArchiverTestCase.test_progress_offc             C   s   | j ddd tjd | j ddd | jdd| j | jd	d
| jd d}| jd| | jd| | jd	d
| jd d}| jd| | jd| dS )ztest that various file status show expected results

        clearly incomplete: only tests for the weird "unchanged" status for nowr   i   r   )r   r;   r  r   z--encryption=repokeyr   z--listz::testrN   zA input/file1zA input/file2z::test1zU input/file1Ni @ i @ )r   r  r  r   r   r  )r   r^   rS   rS   rU   test_file_status6  s    
z!ArchiverTestCase.test_file_statusc             C   s   | j ddd tjd | j ddd | jdd	| j | jd
dd| jd d}tjd}| j ddd tjd|j|j	fd | jd
dd| jd d}| j
d| dS )zbtest that a changed file with faked "previous" mtime still gets backed up in ctime,size cache_moder   s   123)r   r;   r  r   )r   r   z--encryption=repokeyr   z--listz--files-cache=ctime,sizez::test1rN   zinput/file1s   321)nsz::test2zM input/file1N)r   r  r  r   r   rb   r   r>  r4  r?  r  )r   r^   r`  rS   rS   rU   test_file_status_cs_cache_modeH  s    

z/ArchiverTestCase.test_file_status_cs_cache_modec             C   s   | j ddd tjd | j ddd | jdd| j | jdd	d
| jd d}tjd}tjd|jtj	A  | jdd	d
| jd d}| j
d| dS )zetest that a chmod'ed file with no content changes does not get chunked again in mtime,size cache_moder   r   )r   r;   r  r   z--encryption=repokeyr   z--listz--files-cache=mtime,sizez::test1rN   zinput/file1z::test2zU input/file1N)r   r  r  r   r   rb   r   r   r  S_IRWXOr  )r   r^   r`  rS   rS   rU   test_file_status_ms_cache_modeW  s    

z/ArchiverTestCase.test_file_status_ms_cache_modec             C   sv   | j ddd tjd | j ddd | jdd| j | jdd	d
| jd d}| jdd	d
| jd d}| jd| dS )zItest that files get rechunked unconditionally in rechunk,ctime cache moder   r   )r   r;   r  r   z--encryption=repokeyr   z--listz--files-cache=rechunk,ctimez::test1rN   z::test2zA input/file1N)r   r  r  r   r   r  )r   r^   rS   rS   rU   test_file_status_rc_cache_modee  s    
z/ArchiverTestCase.test_file_status_rc_cache_modec             C   s   | j ddd tjd | j ddd trR| j ddd tjtjj| j	dt
j | jdd	| j | jd
dd| jd d}| jd| | jd| tr| jd| | jd
dd| jd ddd}| jd| | jd| tr| jd| dS )z#test that excluded paths are listedr   i   r   )r   r;   r  r  r   z--encryption=repokeyr   z--listz--exclude-nodumpz::testrN   zA input/file1zA input/file2zx input/file3z::test1z	--excludez*/file2zU input/file1zx input/file2Ni @ i @ i @ )r   r  r  r<   r   r   rb   r   r   r   r   r   r   r   r  )r   r^   rS   rS   rU   test_file_status_excludedp  s"    
z*ArchiverTestCase.test_file_status_excludedc             C   s   | j ddd | jdd| j tj| jdd| jd	 d
}d|ksFtd|ksRtd|ks^td|d ksnt|d }|d dkstt|d tstt|d tstt	|d dkstd|kstd S )Nr   i   r   )r   r   z--encryption=repokeyr   z--jsonz::testrN   r  r   r   r  r   r   r  r  r  rQ  r  r  i @ )
r   r   r   r  r  r   rY   rl   r  r   )r   Zcreate_infor   rS   rS   rU   test_create_json  s    z!ArchiverTestCase.test_create_jsonc             C   s   | j ddd tjd | j ddd | jdd| j | jd	| jd
 d}| jd| | jd	| jd d}| jd| | jd	dd| jd d}| jd| | jd	dd| jd d}| jd| | j ddd | jd	dd| jd d}| jd| d S )Nr   i   r   )r   r;   r  r   z--encryption=repokeyr   z::testrN   z::test0z--listz
--filter=Uz::test1z--filter=AMz::test2r   z::test3i @ i @ i  )r   r  r  r   r   r  r  )r   r^   rS   rS   rU   test_create_topical  s    
z$ArchiverTestCase.test_create_topicalc             C   s^   t jdt jj| jd | jdd| j | jd }| jdd|d | jd	|}d
|ksZtd S )Nzsomewhere does not existr   r   z--encryption=repokeyz::testr   z--read-specialrN   rl   z&input/link -> somewhere does not exist)rb   r   r   r   r   r   r   r   )r   r   r^   rS   rS   rU   'test_create_read_special_broken_symlink  s    
z8ArchiverTestCase.test_create_read_special_broken_symlinkc             C   s  | j dd| j | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j d	d
d| jd}| jd| | jd| | jd| | jd| | j d| j}| jd| | jd| | jd| | jd| | jd| | j d	| jd | j d| j}| jd| | jd| | jd| | jd| | jd| | j d| jd t | j d	| jd | j d| j}| jd| | jd| d S )Nr   z--encryption=repokeyr   z::test1z::test2z::test3.checkpointz::test3.checkpoint.1z::test4.checkpointr$  z--listz	--dry-runz--keep-daily=2zKeeping archive: test2zWould prune:     test1z!Keeping archive: test4.checkpointrl   test1r  ztest3.checkpointztest3.checkpoint.1ztest4.checkpointz::test5Z
checkpointZtest5)r   r   r   r  r  )r   r^   rS   rS   rU   test_prune_repository  s:    z&ArchiverTestCase.test_prune_repositoryc             C   s   | j dd| j | j d| jd t | j d| jd t | j ddd| jd	}| jd
| | jd| | j d| j}| jd| | jd| | j dd| jd	 | j d| j}| jd| | jd| d S )Nr   z--encryption=repokeyr   z::test1z::test2r$  z--listz	--dry-runz--keep-daily=2zKeeping archive: test2zWould prune:     test1rl   r;  r  z--save-space)r   r   r   r  r  )r   r^   rS   rS   rU    test_prune_repository_save_space  s    z1ArchiverTestCase.test_prune_repository_save_spacec             C   s   | j dd| j | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j dd	d
| jdd}| jd| | jd| | j d| j}| jd| | jd| | jd| | jd| | j d| jdd | j d| j}| jd| | jd| | jd| | jd| d S )Nr   z--encryption=repokeyr   z::foo-2015-08-12-10:00z::foo-2015-08-12-20:00z::bar-2015-08-12-10:00z::bar-2015-08-12-20:00r$  z--listz	--dry-runz--keep-daily=2z--prefix=foo-z%Keeping archive: foo-2015-08-12-20:00z%Would prune:     foo-2015-08-12-10:00rl   zfoo-2015-08-12-10:00zfoo-2015-08-12-20:00zbar-2015-08-12-10:00zbar-2015-08-12-20:00)r   r   r   r  r  )r   r^   rS   rS   rU   test_prune_repository_prefix  s&    z-ArchiverTestCase.test_prune_repository_prefixc             C   s   | j dd| j | j d| jd t | j d| jd t | j d| jd t | j d| jd t | j dd	d
| jdd}| jd| | jd| | j d| j}| jd| | jd| | jd| | jd| | j d| jdd | j d| j}| jd| | jd| | jd| | jd| d S )Nr   z--encryption=repokeyr   z::2015-08-12-10:00-fooz::2015-08-12-20:00-fooz::2015-08-12-10:00-barz::2015-08-12-20:00-barr$  z--listz	--dry-runz--keep-daily=2z--glob-archives=2015-*-fooz%Keeping archive: 2015-08-12-20:00-fooz%Would prune:     2015-08-12-10:00-foorl   z2015-08-12-10:00-fooz2015-08-12-20:00-fooz2015-08-12-10:00-barz2015-08-12-20:00-bar)r   r   r   r  r  )r   r^   rS   rS   rU   test_prune_repository_glob  s&    z+ArchiverTestCase.test_prune_repository_globc             C   s   | j dd| j | j d| jd t | j d| jd t | j d| jd t | j dd| j}| jd	| | jd
| | jd| d S )Nr   z--encryption=repokeyr   z::test-1z::something-else-than-test-1z::test-2rl   z--prefix=test-ztest-1ztest-2zsomething-else)r   r   r   r  r  )r   r^   rS   rS   rU   test_list_prefix  s    z!ArchiverTestCase.test_list_prefixc             C   s   | j dd| j | jd }| j d|t | j ddd|}| jd| | j d|}| j dd	d
|}| j dd	d|}| j|| | j|| d S )Nr   z--encryption=repokeyz::testr   rl   z--list-formatr  z#--list-format" has been deprecated.z--formatzB{mode} {user:6} {group:6} {size:8d} {mtime} {path}{extra}{NEWLINE}z{mtime:%s} {path}{NL})r   r   r   r  r  ZassertNotEqual)r   test_archiver  output_1output_2output_3rS   rS   rU   test_list_format'  s    
z!ArchiverTestCase.test_list_formatc             C   s   | j dd| j | j ddd| jd t | j ddd| jd t | j d	| j}| j d	d
d| j}| j|| | j d	d| j}| j|d | j d	d
d| j}| j|d | j d	d
d| j}| jd| | jd| d S )Nr   z--encryption=repokeyr   z	--commentz	comment 1z::test-1z	comment 2z::test-2rl   z--formatz{archive:<36} {time} [{id}]{NL}z--shortztest-1
test-2
z{barchive}/ztest-1/test-2/z{name} {comment}{NL}ztest-1 comment 1
ztest-2 comment 2
)r   r   r   r  r  )r   rB  rC  rD  rS   rS   rU   test_list_repository_format3  s    z,ArchiverTestCase.test_list_repository_formatc             C   st   | j ddd | j ddd d | jdd	| j | jd
 }| jd|d | jddd|}d|ksdtd|ksptd S )N
empty_filer   )r   Zamb   ai@B )r   r   z--encryption=repokeyz::testr   rN   rl   z--formatz{sha256} {path}{NL}zJcdc76e5c9914fb9281a1c7e284d73e67f1809a48a497200e046d39ccc7112cd0 input/ambzQe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 input/empty_file)r   r   r   r   )r   rA  r^   rS   rS   rU   test_list_hashB  s    
zArchiverTestCase.test_list_hashc             C   s   | j ddd | j d ttjj| jdd"}|jdd  |jdd  W d Q R X | jd	d
| j | jd }| jd|d | jddd|}d|kst	d|kst	d S )NrG  r   )r   Z
two_chunksr   s   abbai s   baabr   z--encryption=repokeyz::testr   rN   rl   z--formatz'{num_chunks} {unique_chunks} {path}{NL}z0 0 input/empty_filez2 2 input/two_chunks)
r   r   rb   r   r   r   r   r   r   r   )r   r   rA  r^   rS   rS   rU   test_list_chunk_countsL  s    

z'ArchiverTestCase.test_list_chunk_countsc             C   s   | j ddd | jdd| j | jd }| jddd	|d
 | jddd|}|jdd jd\}}}}}t|t|k s|tt|t|k stt|t|kstt|t|kstd S )NZcompressible_filei'  )r   r   z--encryption=repokeyz::testr   z-Clz4rN   rl   z--formatz*{size} {csize} {dsize} {dcsize} {path}{NL}r  r;    )r   r   r   r   rI  r   )r   rA  r^   r   csizeZdsizeZdcsizer   rS   rS   rU   test_list_sizeY  s    
zArchiverTestCase.test_list_sizec             C   s  | j dd%d | jdd| j | jd| jd d	 tj| jd
d| j}|d }t|d dksdttj|d t	sxt|d d dkstd|d kst|d d }tj|d t	st| jd
d| jd }dd |j
 D }t|dkst|d }|d dkst|d dksttj|d  t	s4t| jd
dd!| jd }d"d |j
 D }t|dksnt|d }|d dkst|d# d$kstd S )&Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   rl   z--jsonr   rQ  r  r  r  r  r  r  r  r   r  z--json-linesc             S   s   g | ]}t j|qS rS   )r  r  )r   srS   rS   rU   r  s  s    z3ArchiverTestCase.test_list_json.<locals>.<listcomp>r   r;   r   zinput/file1r   i @ rA  z--format={sha256}c             S   s   g | ]}t j|qS rS   )r  r  )r   rO  rS   rS   rU   r  {  s    r   Z@b2915eb69f260d8d3c25249195f2c8f4f716ea82ec760ae929732c0262442b2bi @ )r   r   r   r  r  r   r   r   r  r  r   )r   Z	list_repor   Zarchive0Zlist_archiver  r   rS   rS   rU   test_list_jsone  s0    zArchiverTestCase.test_list_jsonc             C   s@   | j dd| j | j dd| jdd | j dd| jd dd d S )	Nr   z--encryption=repokeyrl   z--json-linesr   )rj   z--jsonz	::archive)r   r   )r   rS   rS   rU   test_list_json_args  s    z$ArchiverTestCase.test_list_json_argsc             C   s   | j   | jdd| j | jdd| jd ddd}i }x&|j D ]}tj|}|||d	 < q@W |d
 }d|ksrt|d jdst|d }t|d t	st|d dkstt|d t
std S )Nr   z--encryption=repokeyr   z
--log-jsonz::testrN   z--listz--debugtypefile_statusZstatusr   log_messager  Z	levelnameDEBUGmessage)r  r   r   r   r  r  r   r  rY   r  rZ   )r   logZmessagesr   msgrS  rT  rS   rS   rU   test_log_json  s    
zArchiverTestCase.test_log_jsonc             C   s   | j   | jdd| j | jd| jd dd | jddd	d
 tjd
}|j  |jd | jd| jd dd tjd
}|j  |jd d S )Nr   z--encryption=repokeyr   z::testrN   z--debug-profile=create.profdebugzconvert-profilezcreate.profzcreate.pyprofZcumtimez::test2z--debug-profile=create.pyprof)r  r   r   pstatsZStatsZ
strip_dirsZ
sort_stats)r   r  rS   rS   rU   test_debug_profile  s    


z#ArchiverTestCase.test_debug_profilec             C   s>   | j   | jdd| j | jdd| jd d}d|ks:td S )Nr   z--encryption=repokeyz--debugr   z::testrN   z security: read previous location)r  r   r   r   )r   rW  rS   rS   rU   test_common_options  s    z$ArchiverTestCase.test_common_options'  c       	      C   s   |rd| }n
t j|}| jd|d | jdd| j | jd }| jdd||d	 | jd
dd|}|jdd jd\}}}t|t|fS )Nr   r8  )r   r   z--encryption=nonez::testr   z-CrN   rl   z--formatz{size} {csize} {path}{NL}r  r;   rL  )rb   r   r   r   r   r   rI  )	r   Zcompressioncompressibler   r   r   r^   rM  r   rS   rS   rU   
_get_sizes  s    


zArchiverTestCase._get_sizesc             C   s2   | j ddd\}}||kst||d ks.td S )NnoneT)r_     )r`  r   )r   r   rM  rS   rS   rU   "test_compression_none_compressible  s    z3ArchiverTestCase.test_compression_none_compressiblec             C   s2   | j ddd\}}||kst||d ks.td S )Nra  F)r_  rb  )r`  r   )r   r   rM  rS   rS   rU   $test_compression_none_uncompressible  s    z5ArchiverTestCase.test_compression_none_uncompressiblec             C   s&   | j ddd\}}||d k s"td S )NzlibT)r_  g?)r`  r   )r   r   rM  rS   rS   rU   "test_compression_zlib_compressible  s    z3ArchiverTestCase.test_compression_zlib_compressiblec             C   s"   | j ddd\}}||kstd S )Nre  F)r_  )r`  r   )r   r   rM  rS   rS   rU   $test_compression_zlib_uncompressible  s    z5ArchiverTestCase.test_compression_zlib_uncompressiblec             C   s&   | j ddd\}}||d k s"td S )NrK  T)r_  g?)r`  r   )r   r   rM  rS   rS   rU   !test_compression_lz4_compressible  s    z2ArchiverTestCase.test_compression_lz4_compressiblec             C   s"   | j ddd\}}||kstd S )NrK  F)r_  )r`  r   )r   r   rM  rS   rS   rU   #test_compression_lz4_uncompressible  s    z4ArchiverTestCase.test_compression_lz4_uncompressiblec             C   s&   | j ddd\}}||d k s"td S )NlzmaT)r_  g?)r`  r   )r   r   rM  rS   rS   rU   "test_compression_lzma_compressible  s    z3ArchiverTestCase.test_compression_lzma_compressiblec             C   s"   | j ddd\}}||kstd S )Nrj  F)r_  )r`  r   )r   r   rM  rS   rS   rU   $test_compression_lzma_uncompressible  s    z5ArchiverTestCase.test_compression_lzma_uncompressiblec             C   s   | j ddd\}}| jd| j | j ddd\}}| jd| j | j ddd\}}||  kod|kn  snt||d k s~tt||}||kstd S )Nz	auto,zlibT)r_  rr  re  rK  g?)r`  r   r   r   min)r   Z	auto_sizeZ
auto_csizeZ	zlib_sizeZ
zlib_csizeZlz4_sizeZ	lz4_csizeZsmallest_csizerS   rS   rU   "test_compression_auto_compressible  s    
z3ArchiverTestCase.test_compression_auto_compressiblec             C   s2   | j ddd\}}||kst||d ks.td S )Nz	auto,zlibF)r_  rb  )r`  r   )r   r   rM  rS   rS   rU   $test_compression_auto_uncompressible  s    z5ArchiverTestCase.test_compression_auto_uncompressiblec             C   sD   | j dd| j dtjd< | j d| j dtjd< | j d| j d S )Nr   z--encryption=repokeyZnewpassphraseZBORG_NEW_PASSPHRASEzchange-passphraser   rl   )r   r   rb   r   )r   rS   rS   rU   test_change_passphrase  s
    

z'ArchiverTestCase.test_change_passphrasec             C   s"   | j dd| j | j d| j d S )Nr   z--encryption=repokeyz
break-lock)r   r   )r   rS   rS   rU   test_break_lock  s    z ArchiverTestCase.test_break_lockc             C   s   | j   | j d d S )Nz-h)r   )r   rS   rS   rU   
test_usage  s    zArchiverTestCase.test_usagec             C   sj   d| j dkstd| j ddks&td| j ddks:td| j dddksPtd| j ddd	ksftd S )
NZBorgr{   r   Z
Initializer   zpositional argumentsz--epilog-onlyzThis command initializesz--usage-only)r   r   )r   rS   rS   rU   	test_help	  s
    zArchiverTestCase.test_helpc          2   C   s  dd }| j dd| j | j  |d}| j dd| jd d	 | j dd| jd
 d	 trltjtjjd	d tjj| jd}| j	| j|F | j
| jtjj|dd	ddd | j
| jtjj|dd	ddd W d Q R X | j	| jd | | j
| jtjj|d	ddd d}tjj|d	d}tj|}tj|}|j|jksDt|j|jksVt|j|jksht|j|jkszt|r|j|jkst|j|jkst|j|jkstt r|j|j  kodkn  stt|d2}t|d}	|j |	j kstW d Q R X W d Q R X t rd}tjj|d	d}tj|}
tj|}|
j|j  koldkn  sxt|j|jkstt r(d}tjj|d	d}tj|dd}tj|dd}|jtdkst|jtdksttj|jsttj|jsttj|tj|ks(tt r\tjj|d	d}tj|}tj|js\tyd}tjj|d	d}t j! rt j"| jrt#t$t j%|ddgkstt j&|dd kstt j&|dd ksFtnnt$t j%|g kstyt j&|d W n6 t'k
r6 } z|j(t)j*ks&tW Y d d }~X nX dsFtd!W nD t'k
r } z&t+j,j-d#rz|j(t(j.krzn W Y d d }~X nX W d Q R X d S )$Nc             S   s\   t j| j}yt jt j| t W n tk
r6   dS X t j| j}ttk}|oV||kS d S )NF)rb   r   r4  r0  r   r   r  r   )r6  r7  r9  r:  rS   rS   rU   r;  
	  s    z/ArchiverTestCase.test_fuse.<locals>.has_noatimer   z--encryption=repokeyzinput/file1r   z--exclude-nodumpz	::archiverN   z
::archive2r   r  r   T)ignore_bsdflagsignore_xattrsarchive2r   r   rZ  zinput/hardlinkr   zinput/link1r   F)r  r   r   zinput/fusexattrr   z
user.emptyzuser.foos   barz2expected OSError(ENOATTR), but no error was raisedfreebsd)rw  )/r   r   r  r<   rb   r   r   r   r   r  r  r   r   r  r   st_uidst_gidr[  st_atimest_ctimest_mtimerC   r  r   r5  st_inorB   r   S_ISLNKr2  rD   S_ISFIFOr   r   r   r!  rA   	listxattrr  r   r   llfuseZENOATTRrW   r   r  r  )r   r;  rB  r  Zin_fnZout_fnZsti1Zsto1Zin_fZout_fZsti2Zsto2rC  rD  rr   r   rS   rS   rU   	test_fuse	  s    



$*

$
 $zArchiverTestCase.test_fusec                s  | j dd| j | jddd t rJ| jddd tjdd	 tjdd
 | j d| jd d | jddd | j d| jd d tjj| jd}| j	| j|dd tjj|dd tj
 }tdd |D stddh fdd|D kstt rtjj|ddd}tjj|ddd}tjj|ddd}tj|jtj|j  ko\tj|jkn  shtt|dj dkstW d Q R X | j	| j|ddddh t rtjj|ddd}tjj|ddd}tj|jtj|jkstt|dj dkstW d Q R X d S )Nr   z--encryption=repokeyr  s   first)r   Z	hardlink1s   123456zinput/hardlink1zinput/hardlink2zinput/hardlink3r   z
::archive1rN   s   secondz
::archive2r  z-oZversionsc             s   s   | ]}|j d V  qdS )ztest.N)r  )r   r   rS   rS   rU   r   ~	  s    z;ArchiverTestCase.test_fuse_versions_view.<locals>.<genexpr>c                s$   h | ]}t tjj |d j qS )rZ  )r   rb   r   r   r5  )r   r   )r   rS   rU   	<setcomp>	  s    z;ArchiverTestCase.test_fuse_versions_view.<locals>.<setcomp>zhardlink1.00001Z	hardlink2zhardlink2.00001Z	hardlink3zhardlink3.00001rZ  z-e)r   r   r   rC   rb   r   r   r   r   r  r  r  r   r   r}  r   r5  )r   r  filesZhl1Zhl2Zhl3rS   )r   rU   test_fuse_versions_viewn	  s6    
8"z(ArchiverTestCase.test_fuse_versions_viewc              C   s8  | j dd| j | jd | jd\}}|N x>|j D ]*}|jjdr8|j|jd j	 |j}P q8W dsnt
|j  W d Q R X | j dd| jd	d
 tjj| jd}| j| jd |@ tjt}ttjj|| W d Q R X |jjtjkst
W d Q R X | j| jd |dd ttjj||j  W d Q R X d S )Nr   z--encryption=repokeyr   ztestsuite/archiver.pyr;   Fr   z--repairr   )rj   r  z	::archivez-oZallow_damaged_filesr  )r   r   r   r   r  r   r  rr  r  rQ  r   r  rb   r   r   r  r)  r*  r   r   r  r   ZEIOr0  )r   r   r   r  r   r  r
  rS   rS   rU   test_fuse_allow_damaged_files	  s&    
z.ArchiverTestCase.test_fuse_allow_damaged_filesc          .   C   s  | j dd| j | jd | jd | jd | jd tjj| jd}| j| j|dd	( ttj	tjj|ddgks~t
W d Q R X | j| j|d
d	( ttj	tjj|ddgkst
W d Q R X | j| j|d( ttj	tjj|ddgkst
W d Q R X | j| j|d* ttj	tjj|ddgks<t
W d Q R X | j| j|d. ttj	tjj|ddddgkst
W d Q R X | j| j|d& ttj	tjj|g kst
W d Q R X d S )Nr   z--encryption=repokeyZarch11Zarch12Zarch21Zarch22r  z	--first=2z--sort=namez--last=2z--prefix=arch1z--prefix=arch2z--prefix=archz--prefix=nope)r   r   r   rb   r   r   r   r  r!  r  r   )r   r  rS   rS   rU   test_fuse_mount_options	  s$    



,,,.2z(ArchiverTestCase.test_fuse_mount_optionsc                s   t   t   fdd}j  dtjd< jdd| j |  jdjd d	 |  jdjd
 d	 |  jdjd
  |  d S )Nc                 s   t j} x| j| j j D ]~\}}| j|}t|j }| kr  j| t	t
|d }t|dd }x,t||| D ]}j| j| qW q W W d Q R X d S )N)   !   )r:   r   Z
open_indexZget_transaction_id	iteritemsr   r   Zdigestaddr   r   r   r   r  )r   rQ  _r   hashZ
num_blocksnonceZcounter)seenr   usedrS   rU   verify_uniqueness	  s    

zIArchiverTestCase.verify_aes_counter_uniqueness.<locals>.verify_uniquenessrj  r   r   z--encryption=r   z::testrN   z::test.2rr  )rU  r  rb   r   r   r   )r   methodr  rS   )r  r   r  rU   verify_aes_counter_uniqueness	  s    
z.ArchiverTestCase.verify_aes_counter_uniquenessc             C   s   | j d d S )Nr  )r  )r   rS   rS   rU   #test_aes_counter_uniqueness_keyfile	  s    z4ArchiverTestCase.test_aes_counter_uniqueness_keyfilec             C   s   | j d d S )Nr  )r  )r   rS   rS   rU   &test_aes_counter_uniqueness_passphrase	  s    z7ArchiverTestCase.test_aes_counter_uniqueness_passphrasec             C   s   | j   | jdd| j | jd| jd d td | jdd| jd }W d Q R X ttjd}t|d	kr||d	 jd
st	d|kst	d S )Nr   z--encryption=repokeyr   z::testrN   r^   rZ  zdump-archive-itemsr   Z000000_zDone.)
r  r   r   r?   r!  rb   r  r   r  r   )r   r^   
output_dirrS   rS   rU   test_debug_dump_archive_items	  s    
z.ArchiverTestCase.test_debug_dump_archive_itemsc             C   s   | j   | jdd| j | jd| jd d td | jdd| j}W d Q R X ttjd}t|d	krx|d	 jd
s|t	d|kst	d S )Nr   z--encryption=repokeyr   z::testrN   r^   rZ  zdump-repo-objsr   Z	00000000_zDone.)
r  r   r   r?   r!  rb   r  r   r  r   )r   r^   r  rS   rS   rU   test_debug_dump_repo_objs	  s    
z*ArchiverTestCase.test_debug_dump_repo_objsc             C   s   | j dd| j d}t|j }| jd|d | j dd| jd}||ksLt| j dd	| j|d
}||kslttd
d}|j }W d Q R X ||kst| j dd| j|}d|kst| j dd| j|}d|kst| j dd| jd}d|kstd S )Nr   z--encryption=repokeys	   some datar8  )r   rZ  zput-objz
input/filezget-objzoutput/filerZ  z
delete-objZdeletedz	not foundZinvalidz
is invalid)r   r   r   Z	hexdigestr   r   r   r5  )r   r   Zhexkeyr^   r   Z	data_readrS   rS   rU   test_debug_put_get_delete_obj	  s"    z.ArchiverTestCase.test_debug_put_get_delete_objc             C   sN   dd }t jtd| | jdd| jdd W d Q R X tjj| j sJtd S )Nc              W   s   t d S )N)EOFError)rT   rS   rS   rU   	raise_eof
  s    z7ArchiverTestCase.test_init_interrupt.<locals>.raise_eofr   r   z--encryption=repokeyr;   )rj   )	r   r  r   r   r   rb   r   r   r   )r   r  rS   rS   rU   test_init_interrupt
  s    z$ArchiverTestCase.test_init_interruptc             C   s   | j d| jdd d S )Nr   r   )rj   )r   r   )r   rS   rS   rU   $test_init_requires_encryption_option
  s    z5ArchiverTestCase.test_init_requires_encryption_optionc             C   s`   | j dd| j | jr0| j dd| jd dd n,tjtj | j dd| jd  W d Q R X d S )Nr   z--encryption=repokeyz/nestedr   )rj   )r   r   r   r)  r*  r:   ZAlreadyExists)r   rS   rS   rU   test_init_nested_repositories
  s
    z.ArchiverTestCase.test_init_nested_repositoriesc             C   s   t jj| jd}t|d | jdd| jd  t|}|j }W dQ R X dd| jd f}| j	rt| j|dd	i n$t
jtjj | j|  W dQ R X t|}|j }W dQ R X ||kstW dQ R X dS )
zBORG_KEY_FILE=something borg init should quit if "something" already exists.

        See https://github.com/borgbackup/borg/pull/6046r  )ZBORG_KEY_FILEr   z--encryption=keyfiler  Nr  rj   r   )rb   r   r   r   r@   r   r   r   r5  r   r)  r*  rp   r   Errorr   )r   r  r8  beforeargafterrS   rS   rU   %test_init_refuse_to_overwrite_keyfile
  s    

z6ArchiverTestCase.test_init_refuse_to_overwrite_keyfilec             C   s  | j d| j | j b}tj|tj\}}t|||dd}|j}W d Q R X tj| t|||}|j}W d Q R X W d Q R X ||k	st	t
 }xV|j D ]J\}\}	}
}|| \}}}|	|kst	|
|kst	||kst	|j| qW x&|j D ]\}\}	}
}||kst	qW d S )Nr   F)sync)r   r   r   r'   r   r   r   r  Zdestroyr   rU  r  r  )r   r   r   rI   r   Zoriginal_chunksZcorrect_chunksr  rQ  Zrefcountr   rM  Z
o_refcountZo_sizeZo_csizerS   rS   rU   check_cache,
  s$    

zArchiverTestCase.check_cachec             C   s   | j dd| j | j d| jd d | j b}tj|tj\}}t|||dd4}|j  |jj	t
|jj d d  |j  W d Q R X W d Q R X tjt | j  W d Q R X d S )	Nr   z--encryption=repokeyr   z::testrN   F)r  r   )r   r   r   r'   r   r   r   r+  r  Zincrefrl   r  r  r)  r*  r   r  )r   r   r   rI   r   rS   rS   rU   test_check_cacheB
  s    
z!ArchiverTestCase.test_check_cachec             C   s4   | j dd| j | j d| jddd}d|ks0td S )Nr   z--encryption=repokeyr  z--target=asdfr   )rj   zNeed to specify single archive)r   r   r   )r   r^   rS   rS   rU   test_recreate_target_rcN
  s    z(ArchiverTestCase.test_recreate_target_rcc             C   s   | j   | jdd| j | j  | jd }| jd|d | j  | jd| j}| jd|dd	d
d | j  | jd| j}||kstd|kst| jd }| jdd|}d|kstd|kstd|kstd S )Nr   z--encryption=repokeyz::test0r   rN   rl   r  z
input/dir2z-ezinput/dir2/file3z--target=new-archiveznew-archivez::new-archivez--shortr   z
dir2/file2z
dir2/file3)r  r   r   r  r   )r   r   Zoriginal_archiver  listingrS   rS   rU   test_recreate_targetS
  s"    

z%ArchiverTestCase.test_recreate_targetc             C   s   | j   | jddd | jdd| j | jd }| jd|d	 | jd
|ddd | j  | jdd|}d|ksrtd|ks~td|kstd S )Nz
dir2/file3i   r   )r   r   z--encryption=repokeyz::test0r   rN   r  z
input/dir2z-ezinput/dir2/file3rl   z--shortr   z
dir2/file2i @ )r  r   r   r   r  r   )r   r   r  rS   rS   rU   test_recreate_basicg
  s    
z$ArchiverTestCase.test_recreate_basicc             C   s   | j   | jd| jd d | jd| jd d | j  tdh | jd| jd  tjd	jd
kshttjdjd
ks|ttjdjd
ksttjdjd
kstW d Q R X td, | jd| jd  tjd	jdkstW d Q R X d S )Nr   z::test2rN   r  z::testz
input/dir1r^   r   zinput/dir1/hardlinkr   zinput/dir1/subdir/hardlinkzinput/dir1/aaaazinput/dir1/source2r  )	r  r   r   r  r?   rb   r   r  r   )r   rS   rS   rU   test_recreate_subtree_hardlinkst
  s    

z0ArchiverTestCase.test_recreate_subtree_hardlinksc             C   s  t tjj| jdd"}|jdd  |jdd  W d Q R X | jdd| j | jdd	d
| jd d | jd| jd dd | jd| jd ddd}tt	|j
d\}}||kst| jd| jd	d | j  t	| jd| jd ddstt	| jd| jd ddd std S )NZ
large_filer   rH  i     br   z--encryption=repokeyr   z--chunker-paramsz	7,9,8,128z::test1rN   z::test2z--files-cache=disabledrl   zinput/large_filez--formatz{num_chunks} {unique_chunks}rL  r  defaultz--format={size}z{unique_chunks})r   rb   r   r   r   r   r   r   maprI  r   r   r  )r   r   rl   Z
num_chunksZunique_chunksrS   rS   rU   test_recreate_rechunkify
  s    z)ArchiverTestCase.test_recreate_rechunkifyc             C   s   | j ddd | jdd| j | jd| jd dd	d
 | jd| jd ddd}|jd\}}}t|t|ksrt| jd| jd	dd | j  | jd| jd ddd}|jd\}}}t|t|k st||kstd S )Nr_  i'  )r   r   z--encryption=repokeyr   z::testrN   z-Cra  rl   zinput/compressiblez--formatz{size} {csize} {sha256}rL  r  rK  z--recompress)r   r   r   r   rI  r   r  )r   Z	file_listr   rM  Zsha256_beforeZsha256_afterrS   rS   rU   test_recreate_recompress
  s    z)ArchiverTestCase.test_recreate_recompressc                s   t jttdj j}| j  | jdd| j | jd }| jd|d | jddd	d
d| | jd|j	 }t ddd|j
d  }|jd t fdd|D stt fdd|D std S )Nr   r   z--encryption=repokeyz::test0r   rN   r  z--timestampz1970-01-02T00:00:00z	--commentr  r  i  r;   r   z%Y-%m-%dc                s   g | ]}t jd   |qS )zTime \(start\).+ %s)research)r   r  )s_timerS   rU   r  
  s    z<ArchiverTestCase.test_recreate_timestamp.<locals>.<listcomp>c                s   g | ]}t jd   |qS )zTime \(end\).+ %s)r  r  )r   r  )r  rS   rU   r  
  s    )r   nowr   r   Z
astimezoneZtzinfor  r   r   r   Z	utcoffsetstrftimeanyr   )r   Zlocal_timezoner   r  ZdtimerS   )r  rU   test_recreate_timestamp
  s    

z(ArchiverTestCase.test_recreate_timestampc             C   s   | j ddd | jdd| j | jd| jd d | jd	| jd }| jd
| jddd | j  | jd	| jd }||ks~td S )Nr_  i'  )r   r   z--encryption=repokeyr   z::testrN   rl   r  z-nz-ezinput/compressible)r   r   r   r  r   )r   Zarchives_beforeZarchives_afterrS   rS   rU   test_recreate_dry_run
  s    z&ArchiverTestCase.test_recreate_dry_runc             C   s   | j ddd | jdd| j | jd| jd d	 | jd
| jd }| jd| jdd | j  | jd
| jd }||ks|td S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   r  r  z--chunker-paramsr  i @ )r   r   r   r  r   )r   Zinfo_beforeZ
info_afterrS   rS   rU   !test_recreate_skips_nothing_to_do
  s    z2ArchiverTestCase.test_recreate_skips_nothing_to_doc             C   sN   | j dd| j tjj| jd}ddd| f}| j d| jf|dd	d
 d S )Nr   z--encryption=repokeyzlock.exclusiveZpython3z-cz<import os, sys; sys.exit(42 if os.path.exists("%s") else 23)z	with-lockT*   )rL   rj   )r   r   rb   r   r   r   )r   Z	lock_pathr   rS   rS   rU   test_with_lock
  s    zArchiverTestCase.test_with_lockc             C   sN  | j dd| j | jddd | jddd | jddd | jddd | jd	dd | j d
| jd d | j ddd| jd dd}| j  | jd| | jd| | j dd| jd dd}| j  | jd| | jd| | j d| jd dd}| j  | jd| | jd| | j dd| jd dd}| j  | jd| | jd| d S )Nr   z--encryption=repokeyr   r   )r   r  r  r  Zfile5r   z::testrN   r  z--listz--infoz-ezinput/file2zinput/file1zx input/file2zinput/file3zx input/file3zinput/file4zx input/file4zinput/file5zx input/file5)r   r   r   r  r  r  )r   r^   rS   rS   rU   test_recreate_list_output
  s.    z*ArchiverTestCase.test_recreate_list_outputc          
   C   sD   | j dd| j | j d| jd d | j dddd	d| jd
dd d S )Nr   z--encryption=repokeyr   z::testrN   rr  z--firstr  z--lastTr   )rL   rj   )r   r   )r   rS   rS   rU   test_bad_filters
  s    z!ArchiverTestCase.test_bad_filtersc             C   s   | j d }| jd| jdd | j| j}| jdd| j| t|d}|j }W d Q R X |jdt| d	 srt	| j
d
 tj| j
d  }t|d}|j }W d Q R X ||kst	tj| | jdd| j| t|d}|j }W d Q R X ||kst	d S )Nz	/exportedr   z--encryptionr  rI   exportrz	BORG_KEY r  /r   import)r   r   r   r  r   r   r5  r  r-   r   r   rb   r  r  )r   export_filerepo_idr   export_contentskey_fileZkey_contentsZkey_contents2rS   rS   rU   test_key_export_keyfile
  s     

z(ArchiverTestCase.test_key_export_keyfilec       	      C   sD  | j d }| jd| jdd | j| j}| jdd| j| t|d}|j }W d Q R X |jdt| d	 srt	t
| j}t|}|jd tj  W d Q R X ttjj }|j|tj  |j|jkst	t
| j}|jd
 W d Q R X | jdd| j| t
| j}t|}|jd tj  W d Q R X |j|jks@t	d S )Nz	/exportedr   z--encryptionr  rI   r  r  z	BORG_KEY r  rJ   r  )r   r   r   r  r   r   r5  r  r-   r   r:   r   r   r   Zenv_passphraser   rI   ZTestKeyZMockRepositoryZenc_keyZsave_key)	r   r  r  r   r  r   Zrepo_keyZ
backup_keyZ	repo_key2rS   rS   rU   test_key_export_repokey  s(    
z(ArchiverTestCase.test_key_export_repokeyc          	   C   s   | j d }| jd| jdd | j| j}| jddd| j| t|dd	d
}|j }W d Q R X t||ksnt|j	ds|t|j
dstd S )Nz/exported.htmlr   z--encryptionr  rI   r  z	--qr-htmlr  zutf-8)rR   z<!doctype html>z</html>
)r   r   r   r  r   r   r5  r-   r   r  r  )r   r  r  r   r  rS   rS   rU   test_key_export_qr-  s    
z#ArchiverTestCase.test_key_export_qrc             C   s@   | j d }tj| | jd| jdd | jdd| j|td d S )Nz	/exportedr   z--encryptionr  rI   r  )rj   )r   rb   r   r   r   r,   )r   Zexport_directoryrS   rS   rU   test_key_export_directory:  s    

z*ArchiverTestCase.test_key_export_directoryc          !   C   s  | j d }| jd| jdd | jdd| j|td t|d}|jd	 W d Q R X | jrp| jdd| j|d
d n(tjt	 | jdd| j| W d Q R X t|d}|jd W d Q R X | jr| jdd| j|d
d n(tjt
 | jdd| j| W d Q R X d S )Nz	/exportedr   z--encryptionr  rI   r  )rj   rT  zsomething not a key
r   zBORG_KEY a0a0a0
)r   r   r   r,   r   r   r   r)  r*  r#   r"   )r   r  r   rS   rS   rU   test_key_import_errorsB  s    
z'ArchiverTestCase.test_key_import_errorsc             C   s   d}| j d }| jd| jdd | j| jt| | jd tj| jd  }t	|d0}|j
tjd	 | d
  |j
tdj  W d Q R X | jddd| j| t	|d}|j }W d Q R X |dkstd S )N@e294423506da4e1ea76e8dcdf1a3919624ae3ae496fddf905610c351d3f09239z	/exportedr   z--encryptionr  r  r   rT  rL  r  s   abcdefghijklmnopqrsturI   r  z--paperr  zTo restore key use borg key import --paper /path/to/repo

BORG PAPER KEY v1
id: 2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02
 1: 616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d
 2: 737475 - 88
)r   r   r   rV  r   r   r   rb   r  r   r   r   FILE_IDr   rf   r5  r   )r   r  r  r  r   r  rS   rS   rU   test_key_export_paperkeyZ  s    
z)ArchiverTestCase.test_key_export_paperkeyc             C   s   d}| j d| jdd | j| jt| | jd tj| jd  }t|d0}|j	t
jd | d	  |j	td
j  W d Q R X d}| j ddd| j|d d}| j ddd| j|d d}| j ddd| j|d d S )Nr  r   z--encryptionr  r  r   rT  rL  r  s   abcdefghijklmnopqrstus  2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41  02
2 / e29442 3506da 4e1ea7  25f62a 5a3d41 - 02
2 / e29442 3506da 4e1ea7 / 25f62a 5a3d42 - 02
2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02
616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d


737475 88
73747i - 88
73747 - 88
73 74 75  -  89
00a1 - 88
2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02
616263 646566 676869 6a6b6c 6d6e6f 707172 - 6d
73 74 75  -  88
rI   r  z--paper)rN   s   
y
s1   2 / e29442 3506da 4e1ea7 / 25f62a 5a3d41 - 02

y
)r   r   rV  r   r   r   rb   r  r   r   r   r  r   rf   )r   r  r  r   Ztyped_inputrS   rS   rU   test_key_import_paperkeyt  s    z)ArchiverTestCase.test_key_import_paperkeyc          	   C   s   | j ddd | jdd| j | jd| jd d	 | jd
 }| jdd| j|}|dksZtt|d}tj|}W d Q R X d|kstd|kstd|kstd|kstd|kstd S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   z/dumprZ  zdump-manifestrQ   r  r  rS  Z	item_keys	timestampversioni @ )r   r   r   r   r   r   r  r   )r   	dump_filer^   r   r  rS   rS   rU   test_debug_dump_manifest  s    
z)ArchiverTestCase.test_debug_dump_manifestc          	   C   s   | j ddd | jdd| j | jd| jd d	 | jd
 }| jdd| jd |}|dks^tt|d}tj|}W d Q R X d|kstd|kstd|kstd|kstd S )Nr   i   r   )r   r   z--encryption=repokeyr   z::testrN   z/dumprZ  zdump-archiverQ   r  _nameZ_manifest_entryZ_metaZ_itemsi @ )r   r   r   r   r   r   r  r   )r   r  r^   r   r  rS   rS   rU   test_debug_dump_archive  s    
z(ArchiverTestCase.test_debug_dump_archivec             C   s   | j dd| j | j dd| jdd j }|dks6ttj| j dd	| jd
 d}|d d }| j dd| j|j }|d| d kst| j dd| jddj }|dkstd S )Nr   z--encryption=repokeyrZ  zrefcount-objr  r  zkobject 0000000000000000000000000000000000000000000000000000000000000000 not found [info from chunks cache].r   z--jsonz::testrN   r   rQ  zobject z* has 1 referrers [info from chunks cache].Z124Zxyzaz4object id 124 is invalid.
object id xyza is invalid.)r   r   stripr   r  r  )r   r^   r  
archive_idrS   rS   rU   test_debug_refcount_obj  s    z(ArchiverTestCase.test_debug_refcount_objc             C   s(   | j dd}d|kstd|ks$td S )NrZ  r  zCRC implementationPython)r   r   )r   r^   rS   rS   rU   test_debug_info  s    z ArchiverTestCase.test_debug_infoc             C   s>   | j dd| j tdd | j dd| j| j W d Q R X d S )Nr   z--encryption=repokeyr   )Z_BORG_BENCHMARK_CRUD_TESTZ	benchmarkZcrud)r   r   r@   r   )r   rS   rS   rU   test_benchmark_crud  s    z$ArchiverTestCase.test_benchmark_crudc             C   sL  | j   tjd | jdd| j | jdd| j}| jd| | jd| | jd| | jd	| | jd
| | jd| | jd| xdD ]|\}}| jd| j|}|dkst| jd| j|| | jd| j|}||d kst| jdd| j| | jd| j|dd qW | jddd| jdd | jd| jdd | jd| jddd d S )Nzinput/flagfiler   z--encryption=repokeyrS  z--listz[repository]r  Zsegments_per_dirZstorage_quotaappend_onlyadditional_free_spacerQ  2Grepository.append_onlyr  r  r  z--deleter;   )rj   r   zinvalid-optionr  r  r  r  )r  r  z0
)r  rb   r  r   r   r  r   )r   r^   Zcfg_keyZ	cfg_valuerS   rS   rU   test_config  s0    
 zArchiverTestCase.test_configz(GNU tar must be installed for this test.gzipz%gzip must be installed for this test.c             C   s   | j   tjd | jdd| j | jd| jd d | jd| jd dd	 td
 tjddddg W d Q R X | jdddddd d S )Nzinput/flagfiler   z--encryption=repokeyr   z::testrN   z
export-tarz
simple.tarz
--progressr^   rt   xpfz../simple.tarz--warning=no-timestampzoutput/inputT)rt  ru  	ignore_ns)	r  rb   r  r   r   r?   r]   
check_callr  )r   rS   rS   rU   test_export_tar  s    

z ArchiverTestCase.test_export_tarc             C   s   t jdstjd | j  tjd | jdd| j | jd| jd d | jd	| jd d
d}d|kslt	d|ksxt	t
d tjddddg W d Q R X | jdddddd d S )Nr  zgzip is not installedzinput/flagfiler   z--encryption=repokeyr   z::testrN   z
export-tarzsimple.tar.gzz--listzinput/file1
zinput/dir2
r^   rt   r  z../simple.tar.gzz--warning=no-timestampzoutput/inputT)rt  ru  r  )ru   rv   r)  r.  r  rb   r  r   r   r   r?   r]   r  r  )r   rl   rS   rS   rU   test_export_tar_gz   s    



z#ArchiverTestCase.test_export_tar_gzc             C   s   t jdstjd | j  tjd | jdd| j | jd| jd d | jd	| jd d
dd}d|ksnt	d|kszt	t
d tjddddg W d Q R X | jdddddd d S )Nr  zgzip is not installedzinput/flagfiler   z--encryption=repokeyr   z::testrN   z
export-tarz
simple.tarz--strip-components=1z--listzinput/file1
zinput/dir2
r^   rt   r  z../simple.tarz--warning=no-timestampzoutput/T)rt  ru  r  )ru   rv   r)  r.  r  rb   r  r   r   r   r?   r]   r  r  )r   rl   rS   rS   rU    test_export_tar_strip_components  s    



z1ArchiverTestCase.test_export_tar_strip_componentsc             C   s   | j   | jd| jd dd tdh tjdddd	g tjd
jdksNt	tjdjdksbt	tjdjdksvt	tjdjdkst	W d Q R X d S )Nz
export-tarz::testz
output.tarz--strip-components=2r^   rt   r  z../output.tarz--warning=no-timestampr   r   zsubdir/hardlinkr  r  )
r  r   r   r?   r]   r  rb   r   r  r   )r   rS   rS   rU   &test_export_tar_strip_components_links   s    
z7ArchiverTestCase.test_export_tar_strip_components_linksc             C   s   | j   | jd| jd dd tdh tjdddd	g tjd
jdksNt	tjdjdksbt	tjdjdksvt	tjdjdkst	W d Q R X d S )Nz
export-tarz::testz
output.tarz
input/dir1r^   rt   r  z../output.tarz--warning=no-timestampzinput/dir1/hardlinkr   zinput/dir1/subdir/hardlinkzinput/dir1/aaaazinput/dir1/source2)
r  r   r   r?   r]   r  rb   r   r  r   )r   rS   rS   rU   test_extract_hardlinks_tar,  s    
z+ArchiverTestCase.test_extract_hardlinks_tarc             C   s   t | j}d|d | jgd|d gd|gd|d dgd|gd|gd	|gd
|d gdd|dgdd|dgd|gd|gg}x*|D ]"}| j|ddd}d|ksztqzW d S )Nr   z::testr   r   ro  Znewnamerl   rr  r$  r  rI   r  Zexportedr  zchange-passphrasez
break-lockTr   )rL   rj   zAttic repository detected.)rH   r   r   r   r   )r   r   ZcmdsrT   r^   rS   rS   rU   test_detect_attic_repo8  s     






z'ArchiverTestCase.test_detect_attic_repoc             C   s   dd }| j d tjddd | jd| jd | jd	| jd
 d td6 tjtd| | jd| jd
 td W dQ R X W dQ R X dS )z.https://github.com/borgbackup/borg/issues/6063c              _   s   t tjdd S )Nr/  )r   r   r/  )rT   r  rS   rS   rU   r  Q  s    z`ArchiverTestCase.test_do_not_fail_when_percent_is_in_xattr_name.<locals>.patched_setxattr_EACCESr8  s
   input/files   user.attribute%ps   valuer   z-enoner   z::testrN   r^   r   r   )rj   N)	r   r   r   r   r   r?   r   r  r+   )r   r  rS   rS   rU   .test_do_not_fail_when_percent_is_in_xattr_nameM  s    

z?ArchiverTestCase.test_do_not_fail_when_percent_is_in_xattr_namec             C   s   dd }t jt jj| jd tjddd | jd| jd | jd	| jd
 d t	d6 t
jtd| | jd| jd
 td W dQ R X W dQ R X dS )z.https://github.com/borgbackup/borg/issues/6063c              _   s   t tjdd S )Nr/  )r   r   r/  )rT   r  rS   rS   rU   r  a  s    z_ArchiverTestCase.test_do_not_fail_when_percent_is_in_file_name.<locals>.patched_setxattr_EACCESzdir%ps   input/dir%ps   user.attributes   valuer   z-enoner   z::testrN   r^   r   r   )rj   N)rb   r   r   r   r   r   r   r   r   r?   r   r  r+   )r   r  rS   rS   rU   -test_do_not_fail_when_percent_is_in_file_name]  s    
z>ArchiverTestCase.test_do_not_fail_when_percent_is_in_file_namec             C   s  | j ddd | jdd| j | jd| jd d tjtjj| jd	 tjtjj| j d	 | jd
| j}d|kszt	| jd| j}d|kst	| jd
| jd }d|kst	| jd| jd }d|kst	t
d | jd| jd  W dQ R X | jdd dS )zNonce is only used for encrypting new data.

        It should be possible to retrieve the data from an archive even if
        both the client and the server forget the noncer   s   Hello, borg)r   r   z--encryption=repokeyr   z::testrN   r  r  zAll archives:rl   r  zArchive name: test
r^   r   Nzoutput/input)r   r   r   rb   r   r   r   r   r&   r   r?   r  )r   Z	repo_infoZ	repo_listZarchive_infor&  rS   rS   rU   +test_can_read_repo_even_if_nonce_is_deletedl  s     
z<ArchiverTestCase.test_can_read_repo_even_if_nonce_is_deletedc             C   st   | j ddd | jdd| j | jd| jd d tjj| jd	}tj| | jd| jd
 d tjj|spt	dS )a   We should be able to recover if path/to/repo/nonce is deleted.

        The nonce is stored in two places: in the repo and in $HOME.
        The nonce in the repo is only needed when multiple clients use the same
        repo. Otherwise we can just use our own copy of the nonce.
        r   s   Hello, borg)r   r   z--encryption=repokeyr   z::testrN   r  z::test2N)
r   r   r   rb   r   r   r   r   r   r   )r   r  rS   rS   rU   %test_recovery_from_deleted_repo_nonce  s    
z6ArchiverTestCase.test_recovery_from_deleted_repo_nonceN)r^  )r  r  r	  r)  markskipifrC   Zrequires_hardlinksr&   r  r(  r,  r1  rB   r3  rE   rE  rF   rL  rP  r  rV  rg  ri  rl  rq  rs  rt  rv  r{  r|  r}  r  r  unittest
skipUnlessr=   r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r   r   r  r  r  r  r  r  r  r  r  r  r  r  r  r  r   r  r  r  r  r  r  r  r  r  rb   r   r  r  r  r  r  r  r   r!  r#  r%  r&  Zallow_cache_wiper-  r/  r0  r1  r3  r5  r6  r7  r8  r9  r:  r<  r=  r>  r?  r@  rE  rF  rI  rJ  rN  rP  rQ  rY  r\  r]  r`  rc  rd  rf  rg  rh  ri  rk  rl  rn  ro  rp  rq  rr  rs  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  r  rz   Zrequires_gnutarru   rv   Zrequires_gzipr  r  r  r  r  r  r  r  r  r  rS   rS   rS   rU   r
    s  ?		3!!(!		
	


)'

f 

 

-  r
  zno borg.exe availablec                   s   e Zd ZdZdZejddd Zejddd Zejdd	d
 Z	ejddd Z
ejddd Zejddd Z fddZejddd Zejddd Z  ZS )ArchiverTestCaseBinaryzborg.exeTz(does not raise Exception, but sets rc==2c             C   s   d S )NrS   )r   rS   rS   rU   r,    s    z,ArchiverTestCaseBinary.test_init_parent_dirszpatches objectsc             C   s   d S )NrS   )r   rS   rS   rU   r    s    z*ArchiverTestCaseBinary.test_init_interruptc             C   s   d S )NrS   )r   rS   rS   rU   r    s    z0ArchiverTestCaseBinary.test_extract_capabilitiesc             C   s   d S )NrS   )r   rS   rS   rU   r    s    z1ArchiverTestCaseBinary.test_extract_xattrs_errorszLtest_basic_functionality seems incompatible with fakeroot and/or the binary.c             C   s   d S )NrS   )r   rS   rS   rU   r    s    z/ArchiverTestCaseBinary.test_basic_functionalityzBtest_overwrite seems incompatible with fakeroot and/or the binary.c             C   s   d S )NrS   )r   rS   rS   rU   r    s    z%ArchiverTestCaseBinary.test_overwritec                s    t  rtjd n
t j  d S )Nz9test_fuse with the binary is not compatible with fakeroot)rG   r  r.  superr  )r   )	__class__rS   rU   r    s    z ArchiverTestCaseBinary.test_fusec             C   s   d S )NrS   )r   rS   rS   rU   r    s    zEArchiverTestCaseBinary.test_do_not_fail_when_percent_is_in_xattr_namec             C   s   d S )NrS   )r   rS   rS   rU   r    s    zDArchiverTestCaseBinary.test_do_not_fail_when_percent_is_in_file_name)r  r  r	  r   r   r  r.  r,  r  r  r  r  r  r  r  r  __classcell__rS   rS   )r  rU   r    s   r  c                   s   e Zd Z fddZdd Zdd Zdd Zd	d
 Zdd Zdd Z	dd Z
dd Zdd Zdd Zdd Zdd Zdd Zdd Z  ZS )ArchiverCheckTestCasec                sL   t  j  tjtdd* | jdd| j | jd | jd W d Q R X d S )NBUFFER_SIZEr   r   z--encryption=repokeyarchive1rv  )r  r   r   r  r   r   r   r   )r   )r  rS   rU   r     s
    

zArchiverCheckTestCase.setUpc             C   s<  | j ddd| jdd}| jd| | jd| | jd| tjd	jtj | j ddd
| jdd}| jd| | jd| | jd| | j ddd| jdd}| jd| | jd| | j dddd| jdd}| jd| | j dddd| jdd}| jd| | jd| | j dddd| jdd}| jd| | jd| d S )Nr   z-vz
--progressr   )rj   zStarting repository checkz"Starting archive consistency checkzChecking segmentszborg.output.progressz--repository-onlyz--archives-onlyz--prefix=archive2r  z	--first=1rv  z--last=1)r   r   r  loggingZ	getLoggerZsetLevelZNOTSETr  )r   r^   rS   rS   rU   test_check_usage  s(    z&ArchiverCheckTestCase.test_check_usagec             C   s,  | j d\}}|R xB|j D ],}|jjdr|j}|d }|j|j P qW | jd |j  W d Q R X | j	d| j
dd | j	dd| j
dd}| jd	| | j	d| j
dd | j	d
d| j
d dd}| jd| xtdD ]l}| j |\}}|P xH|j D ]2}|jjd r| j||j | j||j P  qW | jd W d Q R X qW tjtdd | jd W d Q R X | j	ddd| j
dd}| jd| | jd| xhdD ]`}| j |\}}|B x:|j D ]$}|jjdr| j||j P qW | jd W d Q R X qW | j	d
d| j
d dd}| jd| d S )Nr  ztestsuite/archiver.pyr;   zshould not happenr   )rj   z--repairr   zNew missing file chunk detectedrl   z--format={health}#{path}{LF}z
::archive1zbroken#rv  r  r   Zarchive3z-vz$Healed previously missing file chunkzAtestsuite/archiver.py: Completely healed previously damaged file!r  )r  rv  )r  rv  )r   r  r   r  r  rr  rQ  Zfailr  r   r   r  Zassert_not_equalr  r   r  r   r   r   )r   r   r   r  Zvalid_chunksZkilled_chunkr^   archive_namerS   rS   rU   test_missing_file_chunk  sP    


z-ArchiverCheckTestCase.test_missing_file_chunkc             C   st   | j d\}}|  |j|jjd  |j  W d Q R X | jd| jdd | jdd| jdd | jd| jdd d S )Nr  r   r   r;   )rj   z--repair)r   rr  r  r  r  r   r   )r   r   r   rS   rS   rU   test_missing_archive_item_chunk  s    z5ArchiverCheckTestCase.test_missing_archive_item_chunkc             C   sn   | j d\}}| |j|j |j  W d Q R X | jd| jdd | jdd| jdd | jd| jdd d S )Nr  r   r;   )rj   z--repairr   )r   rr  rQ  r  r   r   )r   r   r   rS   rS   rU   test_missing_archive_metadata#  s    z3ArchiverCheckTestCase.test_missing_archive_metadatac             C   s   | j d\}}| |jtj |j  W d Q R X | jd| jdd | jddd| jdd}| jd| | jd| | jd| jdd d S )	Nr  r   r;   )rj   z-vz--repairr   rv  )r   rr  r'   MANIFEST_IDr  r   r   r  )r   r   r   r^   rS   rS   rU   test_missing_manifest,  s    z+ArchiverCheckTestCase.test_missing_manifestc             C   s   | j d\}}|0 |jtj}|d }|jtj| |j  W d Q R X | jd| jdd | jddd| jdd}| jd| | jd	| | jd| jdd d S )
Nr  s
   corrupted!r   r;   )rj   z-vz--repairr   rv  )	r   r   r'   r  r  r  r   r   r  )r   r   r   r   corrupted_manifestr^   rS   rS   rU   test_corrupted_manifest7  s    z-ArchiverCheckTestCase.test_corrupted_manifestc             C   s   | j d\}}|R |jtj}|d }|jtj| |j|j}|d }|j|j| |j  W d Q R X | jd| jdd | jddd| jdd}| j	d	| | jd| jdd d S )
Nr  s
   corrupted!r   r;   )rj   z-vz--repairr   rv  )
r   r   r'   r  r  rQ  r  r   r   r  )r   r   r   r   r  chunkZcorrupted_chunkr^   rS   rS   rU   %test_manifest_rebuild_corrupted_chunkD  s    z;ArchiverCheckTestCase.test_manifest_rebuild_corrupted_chunkc       	      C   s   | j d\}}|j}|n |jtj}|d }|jtj| g g dddddd}|j|dd	}|j|}|j||j| |j	  W d Q R X | j
d
| jdd | j
d
d| jdd | j
d| j}| jd| | jd| | jd| d S )Nr  s
   corrupted!foobarz2016-12-15T18:49:51.849711r;   )cmdliner  hostnameusernamer   r  r  s   archive)contextr   )rj   z--repairr   rl   z
archive1.1rv  )r   rI   r   r'   r  r  Zpack_and_authenticate_metadataid_hashencryptr  r   r   r  )	r   r   r   rI   r   r  Zarchive_dictr  r^   rS   rS   rU   'test_manifest_rebuild_duplicate_archiveT  s.    
z=ArchiverCheckTestCase.test_manifest_rebuild_duplicate_archivec             C   s   | j d| jdd t| jdd}|jdd |j  W d Q R X | j d| jdd | j d| jdd | j dd	| jdd | j d| jdd | j d
d| jd dd d S )Nr   r   )rj   T)r   s    01234567890123456789012345678901s   xxxxr;   z--repairr   z	--dry-runz
::archive1)r   r   r:   r  r  )r   r   rS   rS   rU   test_extra_chunksp  s    z'ArchiverCheckTestCase.test_extra_chunksc             G   s  t j| j | jd| jf|  | jd | jd\}}|V xF|j D ]:}|jj	drH|j
d }|j|jd }|j|j| P qHW |j  W d Q R X | jd| jdd | jdd	| jdd}t|jd
 |kst| jddd	| jdd}t|jd
 |kstd|kstd S )Nr   r  ztestsuite/archiver.pyr;   s   1234r   r   )rj   z--verify-dataz, integrity errorz--repairz6testsuite/archiver.py: New missing file chunk detectedr  )ru   r   r   r   r   r   r   r  r   r  r  r   rQ  r  r  r-   r   )r   Z	init_argsr   r   r  r  r   r^   rS   rS   rU   _test_verify_data{  s$    

z'ArchiverCheckTestCase._test_verify_datac             C   s   | j dd d S )Nz--encryptionr  )r  )r   rS   rS   rU   test_verify_data  s    z&ArchiverCheckTestCase.test_verify_datac             C   s   | j dd d S )Nz--encryptionra  )r  )r   rS   rS   rU   test_verify_data_unencrypted  s    z2ArchiverCheckTestCase.test_verify_data_unencryptedc             C   sT   t | jdd*}x|j D ]}|j| qW |j  W d Q R X | jd| jdd d S )NT)r   r   r;   )rj   )r:   r   rl   rr  r  r   )r   r   Zid_rS   rS   rU   test_empty_repository  s
    z+ArchiverCheckTestCase.test_empty_repositoryc             C   s   G dd d}| j d\}}|Z tj|tj\}}t|||0}t|||d|dd}|jj|  |j  W d Q R X W d Q R X | j	d| j
dd	 | j	d
| j
d dd	 d S )Nc               @   s   e Zd Zdd ZdS )zAArchiverCheckTestCase.test_attic013_acl_bug.<locals>.Attic013Itemc          	   S   s   dddddddd dS )NZ1234r      0)s   paths   mtimes   modes   users   groups   uids   gids   aclrS   )r   rS   rS   rU   as_dict  s    zIArchiverCheckTestCase.test_attic013_acl_bug.<locals>.Attic013Item.as_dictN)r  r  r	  r  rS   rS   rS   rU   Attic013Item  s   r  r  z0.13T)r   r   r   r   )rj   rl   z::0.13)r   r'   r   r   r   r   Zitems_bufferr  Zsaver   r   )r   r  r   r   r   rI   r   rS   rS   rU   test_attic013_acl_bug  s    z+ArchiverCheckTestCase.test_attic013_acl_bug)r  r  r	  r   r  r  r  r   r  r  r  r  r  r  r  r  r  r  r  rS   rS   )r  rU   r    s   2		r  c               @   s4   e Zd Zdd Zdd Zdd Zdd Zd	d
 ZdS )ManifestAuthenticationTestc             C   sf   |X t j|t j\}}|jt j|jtjdi i tj	 t
dd jtd |j  W d Q R X d S )Nr;   )days)r  r  rS  r  )r'   r   r   r  r  r  r/   packbr   utcnowr   r  r  r  )r   r   r  rI   rS   rS   rU   spoof_manifest  s    "z)ManifestAuthenticationTest.spoof_manifestc             C   s   | j dd| j t| jdd}|V tj|tj\}}|jtj|j	t
jdi tj tdd jtd |j  W d Q R X tjt | j d| j W d Q R X d S )	Nr   z--encryption=repokeyT)r   r;   )r  )r  r  r  rl   )r   r   r:   r   r'   r   r   r  r  r  r/   r  r   r  r   r  r  r  r)  r*  r    )r   r   r   rI   rS   rS   rU   test_fresh_init_tam_required  s    "z7ManifestAuthenticationTest.test_fresh_init_tam_requiredc             C   sh  | j dd| j | jd t| jdd}| tjtt|j	 t
j|t
j\}}d|_|j|j tj|jd |jt
j}|d= |jt
j|jtj| |j  W d Q R X | j dd	| j}d|kstd
|kst| j dd| j | j dd	| j}d|kstd|kst| j| tjt | j d| j W d Q R X | j ddd| j | j d| j d S )Nr   z--encryption=repokeyarchive1234T)r   Fs   tamrl   z--debugz'Manifest TAM not found and not requiredupgradez--tamzTAM-verified manifestz--force)r   r   r   r:   r   ru   r   r&   r-   rQ  r'   r   r   Ztam_requiredZchange_passphraseZ_passphraser/   ZunpackbZdecryptr   r  r  r  r  r  r   r  r)  r*  r    )r   r   r  rI   r   r^   rS   rS   rU   test_not_required  s0    

z,ManifestAuthenticationTest.test_not_requiredc             C   sZ   | j dd| j | jd | j dd| j t| jdd}| j| | j d| j sVtd S )	Nr   z--encryption=repokeyr  r   z--disable-tamT)r   rl   )r   r   r   r:   r   r  r   )r   r   rS   rS   rU   test_disable  s    

z'ManifestAuthenticationTest.test_disablec             C   sZ   | j dd| j | jd t| jdd}| j| | j dd| j | j d| j sVtd S )	Nr   z--encryption=repokeyr  T)r   r   z--disable-tamrl   )r   r   r   r:   r   r  r   )r   r   rS   rS   rU   test_disable2  s    

z(ManifestAuthenticationTest.test_disable2N)r  r  r	  r  r  r!  r"  r#  rS   rS   rS   rU   r    s
    r  c               @   s,   e Zd Zdd Zdd Zdd Zdd Zd	S )
ArchiveAuthenticationTestc          	   C   sx   t j|t j\}}tjd|g dddtj jtd}|j	|}|j
||j| |tj f|j|< |j  |j  d S )Nr;   rQ   )r  r   r  r	  r
  r  r  )r'   r   r   r/   r  r   r  r  r  r  r  r  r  r  r   r  )r   r   r  r   rI   Zarchive_datar  rS   rS   rU   write_archive_without_tam	  s    
z3ArchiveAuthenticationTest.write_archive_without_tamc             C   s   | j dd| j | jd t| jdd}| | j|d W d Q R X | j dd| j}d	|ks`td
|kslt| j dd| j | j dd| j}d	|kstd|kstd S )Nr   z--encryption=repokeyarchive_tamT)r   archive_no_tamrl   z--format="{name} tam:{tam}{NL}"zarchive_tam tam:verifiedzarchive_no_tam tam:noner   z--archives-tamzarchive_no_tam tam:verified)r   r   r   r:   r   r%  r   )r   r   r^   rS   rS   rU   test_upgrade_archives_tam  s    
z3ArchiveAuthenticationTest.test_upgrade_archives_tamc             C   s   | j dd| j | jd t| jdd}|& | j|d |jtj |j	  W d Q R X | j dd| j | j d	d
| j}d|kst
d|kst
d S )Nr   z--encryption=repokeyr&  T)r   r'  r   z--repairrl   z--format="{name} tam:{tam}{NL}"zarchive_tam tam:verified)r   r   r   r:   r   r%  rr  r'   r  r  r   )r   r   r^   rS   rS   rU   test_check_rebuild_manifest(  s    
z5ArchiveAuthenticationTest.test_check_rebuild_manifestc             C   s   | j dd| j | jd | j dd| j}t| jdd}| | j|d W d Q R X | j dd	| j}d
|ksptd|ks|t| j dd| j | j dd	| j}d
|kstd|kst| j dd| j}||kstd S )Nr   z--encryption=repokeyr&  rl   z--format="{name} {id}{NL}"T)r   r'  z--format="{name} tam:{tam}{NL}"zarchive_tam tam:verifiedzarchive_no_tam tam:noner   z--repair)r   r   r   r:   r   r%  r   )r   Zarchive_id_pre_checkr   r^   Zarchive_id_post_checkrS   rS   rU   test_check_rebuild_refcounts5  s    
z6ArchiveAuthenticationTest.test_check_rebuild_refcountsN)r  r  r	  r%  r(  r)  r*  rS   rS   rS   rU   r$    s   r$  c               @   s\   e Zd ZdZdd Zdd Zdd Zejdd	d
 Z	ejddd Z
dd Zdd ZdS )RemoteArchiverTestCasez__testsuite__:c             C   s   t t| jS )N)r8   r%   r   )r   rS   rS   rU   r   J  s    z&RemoteArchiverTestCase.open_repositoryc          6   C   s<  t jtdd| jg | jdd| j W d Q R X t jtdd| jg0 tjt | jdd| jd  W d Q R X W d Q R X t jtdddg0 tjt | jdd| jd  W d Q R X W d Q R X t	j
j| j}t jtdd|g | jdd| jd  W d Q R X t jtdddd|g | jdd| jd	  W d Q R X d S )
Nextra_test_argsz--restrict-to-pathr   z--encryption=repokeyZ_0z/fooZ_1Z_2Z_3)r   r  r8   r   r   r   r)  r*  r9   rb   r   r   )r   r,  rS   rS   rU   !test_remote_repo_restrict_to_pathM  s    ((z8RemoteArchiverTestCase.test_remote_repo_restrict_to_pathc             C   s   t jtdd| jg | jdd| j W d Q R X tjj| jd}t jtdd|g, t	j
t | jdd| j W d Q R X W d Q R X d S )Nr,  z--restrict-to-repositoryr   z--encryption=repokeyz..)r   r  r8   r   r   r   rb   r   r   r)  r*  r9   )r   r+  rS   rS   rU   'test_remote_repo_restrict_to_repositoryb  s    z>RemoteArchiverTestCase.test_remote_repo_restrict_to_repositoryzonly works locallyc             C   s   d S )NrS   )r   rS   rS   rU   r  k  s    z4RemoteArchiverTestCase.test_debug_put_get_delete_objc             C   s   d S )NrS   )r   rS   rS   rU   r  o  s    z"RemoteArchiverTestCase.test_configc          "   C   s\  | j dd| j | jddd | jddd | jdd	d | jd
dd | jddd | j d| jd d d}td | j dd| jd dd}||kst| jd* | j dd| jd dd}||kstW d Q R X | jd, | j dd| jd dd}||kstW d Q R X | jd, | j dd| jd dd}||ksDtW d Q R X W d Q R X d S )Nr   z--encryption=repokeyzdir/files   test file contents 1)r   z	dir/file2s   test file contents 2zskipped-file1s   test file contents 3zskipped-file2s   test file contents 4zskipped-file3s   test file contents 5r   z::testrN   z)cached responses left in RemoteRepositoryr^   r   z--debugz--strip-componentsr~  r8  r  r  zinput/dir/filer  )r   r   r   r?   r   r  )r   ZmarkerresrS   rS   rU   !test_strip_components_doesnt_leaks  s(    
z8RemoteArchiverTestCase.test_strip_components_doesnt_leakc             C   s<   | j d d }| jd|ddd}| jd| | jd| d	S )
z.https://github.com/borgbackup/borg/issues/6014z-this-repository-does-not-existz::testr  r   T)rj   rL   zthis-repository-does-not-existz$this-repository-does-not-exist::testN)r   r   r  r  )r   r   r^   rS   rS   rU   4test_do_not_mention_archive_if_you_can_not_find_repo  s    zKRemoteArchiverTestCase.test_do_not_mention_archive_if_you_can_not_find_repoN)r  r  r	  r   r   r-  r.  r  r.  r  r  r0  r1  rS   rS   rS   rU   r+  G  s   	r+  c                   sF   e Zd Z fddZdddZdd Zdd	 Zd
d Zdd Z  Z	S )ArchiverCorruptionTestCasec                sF   t  j  | j  | jdd| j tj| jd| jdd d | _d S )Nr   z--encryption=repokeyr  z--jsonr   r   )r  r   r  r   r   r  r  r   )r   )r  rS   rU   r     s    
z ArchiverCorruptionTestCase.setUpr;   c             C   s\   t |dH}|j| tj tdd |j|D }|j| tj |j| W d Q R X d S )Nzr+bc             s   s   | ]}d | V  qdS )   NrS   )r   crS   rS   rU   r     s    z5ArchiverCorruptionTestCase.corrupt.<locals>.<genexpr>)r   r]  rh   SEEK_ENDbytesr5  r   )r   r8  Zamountr   Z	corruptedrS   rS   rU   corrupt  s
    z"ArchiverCorruptionTestCase.corruptc          
   C   sd   | j tjj| jd | jr<| jd| jdd}d|ks`tn$t	j
t | jd| j W d Q R X d S )Nr  r  r   )rj   zfailed integrity check)r7  rb   r   r   r   r   r   r   r   r)  r*  r$   )r   r   rS   rS   rU   test_cache_chunks  s    z,ArchiverCorruptionTestCase.test_cache_chunksc             C   sN   | j d| jd d | jtjj| jd | j d| jd d}d|ksJtd S )Nr   z::testrN   r  z::test1zfiles cache is corrupted)r   r   r7  rb   r   r   r   r   )r   r   rS   rS   rU   test_cache_files  s    z+ArchiverCorruptionTestCase.test_cache_filesc             C   s<  | j d| jd d | j d| jdj }| j d| jd d | j dd| j | j d	| jd
 tjj| jd}ttj|dkst	| j
tjj||d  tjj| jd}td d}|j| |jddttd t|d}|j| W d Q R X | j dd| jd ddd}d|kst	d|ks*t	d|ks8t	d S )Nr   z::test1rN   rl   z--format={id}{LF}z::test2rr  z--cache-onlyr  z--jsonzchunks.archive.dr  z.compactrS  )rR  r   r       rT  z-vz::test3r;   )rj   z,Reading cached archive chunk index for test1z0Cached archive chunk index of test1 is corruptedz-Fetching and building archive index for test1)r   r   r  rb   r   r   r   r   r  r   r7  r   r5  rU  r-   r6  r   r   )r   Z	target_idZchunks_archiveconfig_pathrS  r   r   rS   rS   rU   test_chunks_archive  s$    

z.ArchiverCorruptionTestCase.test_chunks_archivec          
   C   sx   t jj| jd}td d}|j| |jddttd t	|d}|j
| W d Q R X | jd| j}d|ksttd S )	NrS  )rR  r   r   r:  rT  r  zHCache integrity data not available: old Borg version modified the cache.)rb   r   r   r   r   r5  rU  r-   r6  r   r   r   r   r   )r   r;  rS  r   r   rS   rS   rU   test_old_version_interfered  s    

z6ArchiverCorruptionTestCase.test_old_version_interfered)r;   )
r  r  r	  r   r7  r8  r9  r<  r=  r  rS   rS   )r  rU   r2    s   

r2  c               @   s   e Zd Zdd Zdd ZdS )DiffArchiverTestCasec             C   s  | j   | jdd| j | jddd | jddd | jdd	d | jd
dd tjd tjdtjdB  tjd t	 rtjd tj
dd tj
dd tj
dd tj
dd tj
dd t rtjdd tjdd tjdd tjdd | jd| jd d  | jd!d"d | jd#d$d tjd | jd
d%d& d' tjd tjd tjd | jd(d)d tjdtjdB  tjd* tjd t	 rtjd tj
d*d tjd tj
d*d tj
d*d+ tjd | jd,d-d tjd t rtjd tjd.d/ tdd0}|jd1 W d Q R X | jd| jd2 d  | jdd3d4| jd5 d  d6d7 }d8d9 }|| jd:| jd d;d< || jd:| jd d=d>d?d@ || jd:| jd d;dAd< d S )BNr   z--encryption=repokeyZfile_unchanged   )r   Zfile_removed   Zfile_removed2i   file_replacedi   zinput/dir_replaced_with_filei  zinput/dir_removedzinput/dir_replaced_with_linkzinput/link_changedzinput/file_unchangedzinput/link_removedzinput/file_removed2zinput/link_target_removedzinput/emptyz"input/link_target_contents_changedzinput/link_replaced_by_filezinput/file_replacedzinput/hardlink_target_replacedzinput/hardlink_contents_changedzinput/file_removedzinput/hardlink_removedzinput/hardlink_target_removedr   z::test0rN   Z
file_addedi   Zfile_empty_addedr   r  i   )r   Zdir_replaced_with_filei    zinput/dir_addedzinput/link_addedZlink_replaced_by_filei @  zinput/file_addedzinput/hardlink_addedZabs   appended_dataz::test1az--chunker-paramsz16,18,17,4095z::test1bc             S   s  |rdndj d}d| kstdj || ks0td| ks<tdtjkrRd| ksRtd	| ks^td
| ksjtt rd| ks|td| kstd| kstd| kstd| kstd| kst|rdndj d}dj || kstt rdj || kstt r
d| ks
td| kstt r.d| ks.td| ks<td| ksJtt r`d| ks`tt rvd| ksvtt rd| kstd S )NBz{:<19}modifiedrA  z{} input/file_replacedzinput/file_unchangedr  z7[drwxr-xr-x -> -rwxr-xr-x] input/dir_replaced_with_filez#added directory     input/dir_addedz%removed directory   input/dir_removedz&changed link        input/link_changedz$added link          input/link_addedz&removed link        input/link_removedz] input/dir_replaced_with_linkz] input/link_replaced_by_filezinput/link_target_removedz0 Bz{} input/emptyz"{} input/hardlink_contents_changedz"input/link_target_contents_changedz$added       2.05 kB input/file_addedz(added       2.05 kB input/hardlink_addedz*added           0 B input/file_empty_addedz&removed       256 B input/file_removedz*removed       256 B input/hardlink_removedzinput/hardlink_target_removedzinput/hardlink_target_replaced)formatr   rb   r   rB   rC   )r^   can_compare_idsZchangerS   rS   rU   
do_asserts  s@    
zADiffArchiverTestCase.test_basic_functionality.<locals>.do_assertsc             S   s  dd }dd | j dD }|r,dddd	nd
di}||d|ksFtt|d| sZtdtjkr~dddd|d|ks~td
di|d|kstd
di|d|kstt rFd
di|d|kstd
di|d|kstd
di|d|ksttdd |d|D sttd d |d!|D s0tt|d"| sFt|rXdd#d$d	nd
di}||d%|ksttt r||d&|kstt rt|d'| std(d)d*|d+|kstt rd(d)d*|d,|kstd(d$d*|d-|kstd.d/d*|d0|kstt r@d.d/d*|d1|ks@tt r^t|d2| s^tt r|t|d3| s|td S )4Nc                s0    fdd|D }t |dk s"tdd |D S )Nc                s    g | ]}|d   kr|d qS )r   ZchangesrS   )r   r   )r   rS   rU   r  ]  s    zoDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.get_changes.<locals>.<listcomp>r   c             S   s   g | ]}|D ]}|qqS rS   rS   )r   ZchgsetchgrS   rS   rU   r  `  s    )r   r   )r   r   ZchgsetsrS   )r   rU   get_changes\  s    z[DiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.get_changesc             S   s   g | ]}|rt j|qS rS   )r  r  )r   r   rS   rS   rU   r  c  s    zZDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.<listcomp>r  rC  i   i   )rR  addedremovedrR  zinput/file_replacedzinput/file_unchangedr  r  z
drwxr-xr-xz
-rwxr-xr-x)rR  old_modenew_modezinput/dir_replaced_with_filezadded directoryzinput/dir_addedzremoved directoryzinput/dir_removedzchanged linkzinput/link_changedz
added linkzinput/link_addedzremoved linkzinput/link_removedc             s   s(   | ] }|d  dko|d j dV  qdS )rR  r  rL  lN)r  )r   rG  rS   rS   rU   r   |  s    zYDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_asserts.<locals>.<genexpr>zinput/dir_replaced_with_linkc             s   s(   | ] }|d  dko|d j dV  qdS )rR  r  rK  rM  N)r  )r   rG  rS   rS   rU   r   ~  s    zinput/link_replaced_by_filezinput/link_target_removed   r   zinput/emptyzinput/hardlink_contents_changedz"input/link_target_contents_changedrI  i   )rR  r   zinput/file_addedzinput/hardlink_addedzinput/file_empty_addedrJ  r@  zinput/file_removedzinput/hardlink_removedzinput/hardlink_target_removedzinput/hardlink_target_replaced)r   r   r  rb   r   rB   rC   )r^   rE  rH  Zjoutputr  rS   rS   rU   do_json_asserts[  sH    

zFDiffArchiverTestCase.test_basic_functionality.<locals>.do_json_assertsr  Ztest1aTZtest1br;   )rj   Fz--json-lines)r  r   r   r   rb   r   r   r   S_IFDIRrB   r   rC   r   r  rmdirS_IFREGr   r   )r   r   rF  rO  rS   rS   rU   r    sl    













BMz-DiffArchiverTestCase.test_basic_functionalityc             C   s  | j dd| j | jddd | jddd | jdd	d | jd
dd | j d| jd d tjd tjd tjd tjd | jddd | jd
dd | jddd | jddd | j d| jd d | j dd| jd d}ddddd
dg}tdd t||j D std S )Nr   z--encryption=repokeyZa_file_removed   )r   Zf_file_removed   Zc_file_changedr:  Ze_file_changedr  r   z::test0rN   zinput/a_file_removedzinput/f_file_removedzinput/c_file_changedzinput/e_file_changedi   i   Zb_file_addedr?  Zd_file_addedr@  z::test1r  z--sortr;  c             s   s   | ]\}}||kV  qd S )NrS   )r   xr   rS   rS   rU   r     s    z8DiffArchiverTestCase.test_sort_option.<locals>.<genexpr>)	r   r   r   rb   r  r  zipr   r   )r   r^   r  rS   rS   rU   test_sort_option  s.    



z%DiffArchiverTestCase.test_sort_optionN)r  r  r	  r  rW  rS   rS   rS   rU   r>    s    Rr>  c              C   s<  t  } | jddddgd}|j| jks*t|jddgks<t|jdksJt|jd	ksXt| jddd
dgd}|jddgks~t| jddddgd}|jddgkst| jddd
dgd}|jddgkst| jddd
dgd}|jddgkst| jddddgd}|j| jkst| jddgd}|j| jks8td S )Nrp   Zservez--restrict-to-path=/p1z--restrict-to-path=/p2zborg serve --info --umask=0027z/p1z/p2   r  z--restrict-to-repository=/r1z--restrict-to-repository=/r2z/r1z/r2zborg serve --restrict-to-path=/z%borg serve --restrict-to-repository=/z-borg serve --restrict-to-repository=/r1/belowz borg init --encryption=repokey /z-BORG_HOSTNAME_IS_UNIQUE=yes borg serve --info)	r   Zget_argsfuncZdo_server   Zrestrict_to_pathsumask	log_levelZrestrict_to_repositories)rK   rT   rS   rS   rU   test_get_args  s2    
r\  c              C   sp   dd } | dddgdddd	d
gs&t | dgddg s<t | ddgddg sTt | ddgddg slt d S )Nc             S   sT   dd | D }dd |D }t jt|t|}t jt|t|}||ksPt|S )Nc             S   s   g | ]}|qS rS   rS   )r   r   rS   rS   rU   r    s    z<test_compare_chunk_contents.<locals>.ccc.<locals>.<listcomp>c             S   s   g | ]}|qS rS   rS   )r   r   rS   rS   rU   r    s    )r   Zcompare_chunk_contentsiterr   )r  r  Zchunks_aZchunks_bZcompare1Zcompare2rS   rS   rU   ccc  s    z(test_compare_chunk_contents.<locals>.cccs   1234s   567As   bC   1s   23s   4567Ar     Cs   12345s   56s   65s   565)r   )r^  rS   rS   rU   test_compare_chunk_contents  s    ra  c               @   s0   e Zd Zedd Zdd Zdd Zdd Zd	S )
TestBuildFilterc             C   s   d S )NrS   )r  ZmatchedrS   rS   rU   peek_and_store_hardlink_masters  s    z/TestBuildFilter.peek_and_store_hardlink_mastersc             C   sf   t  }|jtdgtj tj|| jd}|tdds<t	|tddsNt	|tdd sbt	d S )NZincludedr   )r   zincluded/filezsomething else)
r3   r  r4   r2   ZIncluder   build_filterrc  r5   r   )r   matcherr  rS   rS   rU   
test_basic  s    zTestBuildFilter.test_basicc             C   s0   t dd}tj|| jd}|tdds,td S )NT)fallbackr   Zanything)r   )r3   r   rd  rc  r5   r   )r   re  r  rS   rS   rU   
test_empty#  s    
zTestBuildFilter.test_emptyc             C   sl   t dd}tj|| jdd}|tdd s0t|tdd sDt|tddsVt|td	dshtd S )
NT)rg  r;   )Zstrip_componentsZshallow)r   zshallow/zdeep enough/filezsomething/dir/file)r3   r   rd  rc  r5   r   )r   re  r  rS   rS   rU   r  (  s    
z%TestBuildFilter.test_strip_componentsN)r  r  r	  staticmethodrc  rf  rh  r  rS   rS   rS   rU   rb    s   rb  c               @   s   e Zd Zedd Zejdd Zejdd Zejdd Z	ejd	d
 Z
ejdd Zdd Zejjddejjdd dd ZdS )!TestCommonOptionsc             C   sv   | ddddd | dddd	d
dd | dddd	ddd | dddddg d | dddddd | ddt dddd d S )Nz-hz--helpr{   zshow this help message and exit)actionr{   z
--criticalr[  r  store_constcriticalwarning)destr{   rk  constr  z--errorerrorz--appendr  ZTOPIC)ro  r{   rk  metavarr  z-pz
--progressprogress
store_true)ro  rk  r{   z--lock-wait	lock_waitNr;   z(default: %(default)d).)ro  rR  rr  r  r{   )rI  )Zadd_common_optionrS   rS   rU   define_common_options2  s    z'TestCommonOptions.define_common_optionsc             C   s&   t jdddd}tj| jdd|_|S )	Nr  ztest parserF)progdescriptionadd_help_level0_level1)Zsuffix_precedence)r{  r|  )argparseArgumentParserr   ZCommonOptionsrw  common_options)r   parserrS   rS   rU   basic_parser?  s    
zTestCommonOptions.basic_parserc             C   s   |j dddS )Nzrequired argumentsz	<command>)titlerr  )Zadd_subparsers)r   r  rS   rS   rU   
subparsersF  s    zTestCommonOptions.subparsersc             C   s   |j j|ddd |S )Nr{  T)Zprovide_defaults)r  add_common_group)r   r  rS   rS   rU   r  J  s    zTestCommonOptions.parserc             C   s    t jddd}|jj|d |S )NFr  )rz  rx  r|  )r}  r~  r  r  )r   r  common_parserrS   rS   rU   r  O  s    zTestCommonOptions.common_parserc          	      sH   |j d|gddddtjd}|jdd |jd	d
dd  fdd}|S )N
subcommandFr  r  baz)parentsrz  ry  epilogr{   Zformatter_classi  )rY  z--append-onlyr  rt  )ro  rk  c                 s&   t |   j| } jj| t|S )N)r   rk   r  Zresolvevars)r   rT   )r  rS   rU   parse_vars_from_line]  s    
zDTestCommonOptions.parse_vars_from_line.<locals>.parse_vars_from_line)Z
add_parserr}  ZRawDescriptionHelpFormatterZset_defaultsadd_argument)r   r  r  r  Z	subparserr  rS   )r  rU   r  U  s    
z&TestCommonOptions.parse_vars_from_linec             C   s   |dg ddddkst |dddg ddddd	d
ks<t tjt |dd W d Q R X |dddddddddgddddd	d
kst d S )Nz--errorr;   rq  F)r  ru  r[  rs  r  z
--criticalrm  i  )r  ru  r[  rs  r  rY  z--append-onlyz--append=fooz--appendr  r  r  rn  )r   r)  r*  r`   )r   r  rS   rS   rU   test_simplee  s*    
zTestCommonOptions.test_simplepositionr  r  bothzflag,args_key,args_value-prs  T--lock-wait=3ru  rb  c             C   s`   g }|d
kr|j | |j d |dkr2|j | g dddddd	}|||< || |ks\td S )Nr  r  r  r  r;   rn  Fi  )r  ru  r[  rs  r  rY  )r  r  )r  r  )r  r   )r   r  r  flagZargs_keyZ
args_valuer   r  rS   rS   rU   test_flag_position_independence  s    


z1TestCommonOptions.test_flag_position_independenceN)r  r  r  r  rs  Tr  ru  rb  )r  r  )r  r  r	  ri  rw  r)  fixturer  r  r  r  r  r  r  parametrizer  rS   rS   rS   rU   rj  1  s    rj  c            	   C   s4   t ddksttjtj t d W d Q R X d S )NZ50M2   i  r   Z5Mi@B i)r   r   r)  r*  r}  ZArgumentTypeErrorrS   rS   rS   rU   test_parse_storage_quota  s    r  c                 sF   t ddj } t ddj }i d fdd	  d| t d|i S )	z0
    Return dict mapping command to parser.
    rp   )rx  ZborgfsNc                s   i }xJ|j D ]@}|jd k	rdt|jkrx"|jj D ]\}}||| | < q4W qW |d k	rb|j| | rp| rpd S x0t|j D ] \}} |d || ||< q~W d S )NZSubParsersActionrL  )Z_actionschoicesrZ   r  r  updater!  )r   r  r   Zextra_choicesr  rk  r   command)discover_levelparsersrS   rU   r    s    

z'get_all_parsers.<locals>.discover_levelrQ   )N)r   Zbuild_parser)r  Zborgfs_parserrS   )r  r  rU   get_all_parsers  s    r  zcommand, parserc             C   s   t |jtr|jjstd S )N)rY   r  r0   Zrstr   )r  r  rS   rS   rU   test_help_formatting  s    r  ztopic, helptextc             C   s   t t|std S )N)rZ   r1   r   )ZtopichelptextrS   rS   rU   test_help_formatting_helptexts  s    r  )r}  r   rh   r  r  rb   r[  r   ru   r-  r   r]   rW   r   r  r  r  Zbinasciir   r   Zconfigparserr   r   r   r   Zhashlibr   r	   r
   Zunittest.mockr   r)  r  ImportErrorrp   Zborg.helpersrQ   r   r   r   r   r   r   r   r   rK   r   r   r   r   r   r   Z	constantsZcrypto.low_levelr   r   Z
crypto.keyr   r   r   r   r    r!   Zcrypto.keymanagerr"   r#   Zcrypto.file_integrityr$   r%   r&   r'   r(   r)   r*   r+   r,   r-   r.   r/   Znanorstr0   r1   r   r2   r3   r4   r  r5   Zlockingr6   Zloggerr7   Zremoter8   r9   r   r:   r<   r=   r>   r?   r@   rA   rB   rC   rD   rE   rF   rG   ZupgraderrH   rI   r   r  r   r   __file__r   rs   rz   Z	BORG_EXESFileNotFoundErrorr  r   r   r   r  r  r   r   r   r
  r  r  r  r  r$  r+  r2  r>  r\  ra  rb  rj  r  r  r  rl   r  r  r  r  rS   rS   rS   rU   <module>   s    1	 &?                       0+ tL@JJ s(!k 