!&h’ UddlZddlZddlZddlmZddlmZddlmZm Z m Z m Z ddl Z ddl mZddlmZmZddlmZddlmZmZddlmZmZmZdd lmZmZmZdd lm Z dd l!m"Z"dd l#m$Z$m%Z%dd l&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-ddl.m/Z/gdZ0e jjbjde jjbjfe jjbjhhZ5eGddZ6de+dede7fdZ8de+dede7fdZ9de:ee:e;e;ffdede de fdZ?de@de fd ZAde+de e<de+de:e e;e jfe e+gerlistintr?t/var/www/pru.catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/torch/ao/quantization/fx/utils.pyr4r4JssJ//S%T "2233..'+CH~5## LS!*/!4/BF*HT#Y,?FCG+Xd3i-@GrMr4nodeargreturncd}d|jvr|jdjdd}|*|t|jkr|j||ury|jjd|uS)zReturns if node arg is weightNtarget_dtype_info weight_indexTweightmetagetlenargskwargs)rOrPrTs rNr0r0XsqLdii'yy!4599.$O  3tyy> ) IIl #s * ;;??8 $ ++rMcd}d|jvr|jdjdd}|*|t|jkr|j||ury|jjd|uS)zReturns if node arg is biasNrS bias_indexTbiasrV)rOrPr]s rNr/r/fsqJdii'YY2377 dK  TYY ' IIj !S ( ;;??6 "c ))rMcustom_module_mappingct}tjtjtjfD]2}|j |i}t|j }||z}4t|S)aGet all the unique custom module keys in the custom config dict e.g. Input: { QuantType.STATIC: { CustomModule1: ObservedCustomModule }, QuantType.DYNAMIC: { CustomModule2: DynamicObservedCustomModule }, QuantType.WEIGHT_ONLY: { CustomModule3: WeightOnlyObservedCustomModule }, } Output: # extract the keys across all inner STATIC, DYNAMIC, and WEIGHT_ONLY dicts [CustomModule1, CustomModule2, CustomModule3] )rIr STATICDYNAMIC WEIGHT_ONLYrXkeysrJ)r_float_custom_module_classes quant_modequant_mode_custom_module_config quant_mode_custom_module_classess rNr&r&tsy.-0E ''):):I4WI>>: rMprefixcljdddtjjffd }|S)N._modulecdtffd }d}||}t||r|dz }||}t||r|S)Nic t|zSN)rD)rrs rN get_attr_namezOget_new_attr_name_with_prefix..get_new_attr_name..get_attr_namesCF? "rMrr)rKhasattr)rrr attr_namers rNget_new_attr_namez8get_new_attr_name_with_prefix..get_new_attr_namesQ #S # !!$ fi( FA%a(Ifi(rM)replacerjruModule)rrs` rNr(r(s/ ^^C %F %((//  rMc|g}|g}|r|j}t|jt|jj z}|D]i}t |t s|jdk(ry|j||jdk(r|jtk(rY|j|k|r|S)aStarting from a target node, trace back until we hit inpu or getattr node. This is used to extract the chain of operators starting from getattr to the target node, for example def forward(self, x): observed = self.observer(self.weight) return F.linear(x, observed) collect_producer_nodes(observed) will either return a list of nodes that produces the observed node or None if we can't extract a self contained graph without free variables(inputs of the forward function). placeholderN call_function) poprJrZr[values isinstanceropappendtargetgetattr)rOnodesfrontierall_argsrPs rNr"r"sFEvH ||~ ?T$++*<*<*>%?? %Cc4(vv& LL FFo-#**2G$ %  LrMrootproducer_nodesct|dkDsJd|jt}ifd}|D]}|j|||<|j ||dt ||}|S)a,Construct a graph module from extracted producer nodes from `collect_producer_nodes` function Args: root: the root module for the original graph producer_nodes: a list of nodes we use to construct the graph Return: A graph module constructed from the producer nodes rz'list of producer nodes can not be emptyc"t|fdS)Nc|SrrL)rOenvs rNzDgraph_module_from_producer_nodes..load_arg..s s4yrM)r)ars rNload_argz2graph_module_from_producer_nodes..load_argsq011rM)rYreverser node_copyoutputr)rrgraphr producer_node graph_modulers @rNr,r,s ~  "M$MM " GEC2(F "__]HEMF LL.,-.tU+L rMrct|S)z Returns the unique device for a module, or None if no device is found. Throws an error if multiple devices are detected. )r)rs rNr!r!s ) 00rMrvaluec(t|}||}t|}t|tjr|j j ntj||}|j|||jd|}|S)z Given a value of any type, creates a getattr node corresponding to the value and registers the value as a buffer to the module. )deviceget_attr) r(r!rrjTensordetachclonetensorregister_buffer create_node) rrrrrrr new_value attr_nodes rNr#r#s6f=!&)I )& 1F eU\\ *   \\% /  9i0!!*i8I rMmodulescachec|r ||vr||Sd}t|tsd}n|jdk(rd}n|jdk(rQt|jtsJt ||jr{t |jd||}n_|jdk(rd}nL|jdk(r8|jtjurt |jd||}n|jdk(rd}n|jtur|jdd vrd}n|jd k(r|jd k(rd}nd}|jD]}t|tr?|D]9}t|tst |||}|xs| }|s+| }|r|||<|ccSnGt|trn6t|tr$t |||}|xs| }|r| }|r|||<|cSd}| }|r|||<|S) z If we know for sure that all of this node's args have no tensors (are primitives), return True. If we either find a tensor or are not sure, return False. Note: this function is not exact. FTr call_modulerrrr)ndimshape call_methodsize) rrrrrDrr rZoperatorgetitemrrJrK) rOrrresultfound_one_tensorrPlist_el!this_list_el_args_have_no_tensorsthis_arg_args_have_no_tensorss rNr r s T{ F dD ! M ! M !$++s+++ &wt{{'; <2499Q<%PF M ! O # x7G7G(G.tyy|WeL J   DIIaL4E$E M !dkkV&; 99* *C#t$"*G!'409'7ER:,<, AA),)9%9F$.4d #)M#*$C%c4(4QWe51(8(99%(%5!5 *0E$K% '+$))FU* *V d MrMcRttdt|jS)z2 Returns all node arg indices after first r)rJrangerYrZ)rOs rNrrhs aTYY( ))rM arg_indicesc:dtdttffd }|S)zu Constructs a function that takes a node as arg and returns the arg_indices that are valid for node.args rOrQc`Dcgc]}|t|jks|c}Scc}wr)rYrZ)rOrrs rNarg_indices_funcz)return_arg_list..arg_indices_funcus&&=a!c$))n*<===s++)rrJrK)rrs` rNr3r3os" >t>S > rMr.z op targetr masked_fillpermuterepeatreshaper transpose unsqueeze unsqueeze_viewr1r%cvt|j|j}tj |t S)z Returns a dict with of non float tensor types as keys and values which correspond to a function to retrieve the list (which takes the node as an argument) )r.rrr1rXr%)rOinfos rNr)r)s+ DGGT[[ )D " & &t^ < > @!   $ ! ; ; @ @ B#   "> ! ; ;#   !7 77rM named_modulesqconfigqhandlerct||}||t|tjjj j jsJt|tjjxrt|xr|jSt|tjjjjS)zD Return whether this refers to the custom module LSTM flow. ) _get_modulerrjao quantizationfxquantize_handlerQuantizeHandlerruLSTMris_custom_module quantizablerOrrrmods rN_is_custom_module_lstmrs dM *Cx3(EHH$9$9$<$<$M$M$]$]^^^ sEHHMM * ,27; ,))+ #uxx{{66;;<|j|5|jtj|df}t ||||}ddd|j5|jtj|df}ddd|j5|jtj|df}t ||||}ddd|j5|jtj|df} t | |||} ddd|j 5|jt || gf} ddd|j 5|jt || gf} dddt |jjD] } | k7s | |k7s| j| "t| S#1swYxYw#1swYdxYw#1swY'xYw#1swYxYw#1swYxYw#1swYxYw)a8 Insert DeQuantStubs after each internal output node of custom module LSTM. Custom module LSTM outputs are nested tuples of the structure (output, (hidden0, hidden1)), Since we cannot dequantize a tuple as a whole, we must first break down the tuple into its components through `getitem`. This function transforms the graph as follows: (1) Split the LSTM node into (output, (hidden0, hidden1)) (2) Insert a DeQuantStub after each internal node (3) Recombine the DeQuantStubs into the same structure as before (4) Reroute all consumers of the original LSTM node and its sub-nodes (e.g. lstm[0]) Before: lstm_output | v original_user(s) After: lstm_output / \ / (getitem) \ / \ v v output hidden | / \ (DeQuantStub) (getitem) | / \ v v v output_dq hidden0 hidden1 | | | | (DeQuantStub) (DeQuantStub) | | | | v v | hidden0_dq hidden1_dq | \ / | (tuple) | \ / | v v | hidden_dq \ / \ (tuple) / v v lstm_output_dq | v original_user(s) For step (4), reroute all users of the original LSTM node(s) as follows: lstm_output -> lstm_output_dq lstm_output[0] -> output_dq lstm_output[1] -> hidden_dq lstm_output[1][0] -> hidden0_dq lstm_output[1][1] -> hidden1_dq Return the node `lstm_output_dq`. rNr) r rrrrrFrJrrdreplace_input_with_reroute_tuple_getitem_pattern)rOrrrr output_dqhiddenhidden0 hidden0_dqhidden1 hidden1_dq hidden_dqlstm_output_dqrs rN3_insert_dequant_stubs_for_custom_module_lstm_outputr2sB   t $N$$X%5%5ayA( uM N   y )B$$X%5%5ayAB   v &P%%h&6&6 D)'5-O P   z *P%%h&6&6 D)'5-O P   z *L''Z0H/JK L   y )O,,Ui5K4MNOTZZ__&': 6>dfn  # #D. 9:#5) 7NNBBPPPP LLOOsG1G#G!1G."1G;,HHG!G+.G8;HHHc fd}fd}d}d dttdttf fd }|||g ||||g||||g |||gg}|D]}||}||cSy) aC Given an argument of a node, if the argument refers to the path through which the node is a consumer of custom module LSTM, return the custom module LSTM node, or None otherwise. This is used to determine whether a node is a consumer of custom module LSTM, and, if so, skip inserting input observers for this node. This is because custom module LSTM produces quantized outputs, so inserting an input observer for the consumer of custom module LSTM would unnecessarily quantize the outputs again. lstm -> consumer In practice, however, custom module LSTM outputs a tuple (output, (hidden0, hidden1)) with DeQuantStubs attached to each internal node (see `_insert_dequant_stubs_for_custom_module_lstm_output`). This tuple can be consumed in one of four ways: lstm -> getitem -> DeQuantStub -> consumer # consume lstm[0] lstm -> getitem -> getitem -> DeQuantStub -> tuple -> consumer # consume lstm[1] lstm -> getitem -> getitem -> DeQuantStub -> consumer # consume lstm[1][0] or lstm[1][1] lstm -> getitem -> DeQuantStub -> tuple -> consumer # consume lstm Thus, we must match against the above patterns instead of simply checking the parent node to determine whether this node is a consumer of a custom module LSTM. c8tt|tSr)rrrrrs rNmatch_dqz=_maybe_get_custom_module_lstm_from_node_arg..match_dqs+a7EErMct|Sr)rrs rN match_lstmz?_maybe_get_custom_module_lstm_from_node_arg..match_lstms%a77rMc^|jdk(xr|jtjk(SNr)rrrrrs rN match_getitemzB_maybe_get_custom_module_lstm_from_node_arg..match_getitems%tt&G188x7G7G+GGrMcJ|jdk(xr|jtk(Sr#)rrrFr$s rN match_tuplez@_maybe_get_custom_module_lstm_from_node_arg..match_tuplestt&<188u+<._match_patternsk !-0 "HAu83}%))K'q ! Aq A "rMN)rJrrr) rPrrr!r%r,all_match_patternsp matched_noder's `` @rN+_maybe_get_custom_module_lstm_from_node_argr0s8F8H=d8n$$ =*- h }jI =-< h z:   %a(  #   rMc dtdttdttdtttdtttttdfff fd g}t}|j D]} |gg|||D]}|d}|d }|j d k(r|jtk(sJ|j d k(r|jtjk(sJ|jd }|jd|}t|jjD]} | j||y ) a Search for patterns where N consecutive `tuple` call_function nodes are followed by N consecutive `getitem` call_function nodes that are "reverses" of the `tuple` nodes. If we find this pattern, reroute the consumers of the last `getitem` to skip these N `tuple` and `getitem` nodes. Before: a b c | \ / \ tuple \ / tuple | getitem(1) | getitem(0) | d After: b | d rO index_stackcurrent_patternmatched_patternsseen.c t|dk(rBt|dkDr4|jtj||j|t |f}||vry|j ||j D]}|jdk(rh|jtk(rUt|jdD]9\}}||k(s |j||j| |||||;z|jdk(s|jtjk(st|dkDs|jd|dk(s|j|j| ||||||S)aP Traverse the graph recursively to match for the N-tuple - N-getitem patterns, starting at the given node. We use a stack to keep track of the expected `getitem` indices, since these are reversed from the `tuple` indices. In the above example, the stack after (b -> tuple -> tuple) will be [0, 1], which will be popped by getitem(1) first and then by getitem(0). TODO: traverse upwards from the output and handle the case when tuple is not a separate node, e.g. graph.call_function(operator.getitem, args=(a, (b, c))) rNrrr)rYrrclearrFaddrrrr*rZrrr) rOr2r3r4r5staterruser_arg find_patternss rNr;z5_reroute_tuple_getitem_pattern..find_patternssa& { q S%9A%=  # #DIIo$> ?  ! ! #u[)* D=  JJ Dww/)dkkU.B#,TYYq\#:KAx4'#**1-'..t4% +@PRV O+ x?O?O0O{#a'yy|{26#)'..t4% +@PRV " rMrrrrN)rrJrKrIrFrrrrrrZrrdr) rr4r5rOpattern first_tuple last_getitemlast_getitem_index new_inputrr;s @rNrrsZ8/ / #Y/ d/ tDz* / %eCHo-./ / d*,.1eD <dB$4d;< $ =aj r{ ~~0[5G5G55PPP OO .##x'7'77  8*..q1$$Q'(:; ++0023 =D  # #L) < = =rMactivation_post_processcbt|tr|St|tsJ|jS)z If `activation_post_process` is an observer, return the observer. If `activation_post_process` is a fake quantize, return the internal observer. )rrr rA)rAs rN*_get_observer_from_activation_post_processrC;s3)<8&&13CDDD&>>>rMdtype_with_constraints is_activationc4dtttfdtdtdt ffd } |j y|r jn j}|rdnd}d}|8|}t|sJ|j |j k7ry||||}|S) a Return whether `qconfig` satisfies the following constraints from the backend, specified through the activation and weight DTypeWithConstraints. 1. QConfig specified a quantization range that falls within the backend's, if any 2. QConfig specified a min scale value that is >= the backend's, if any 3. QConfig specified a FixedQParamsObserver or FixedQParamsFakeQuantize that has scale and zero point that match the backend's, if any If `is_activation` is True, we check `qconfig.activation`, else we check `qconfig.weight`. If `qconfig` or `dtype_with_constraints.dtype` is None, or the dtypes do not match, return True. rArD debug_stringrQct|}t|dd}t|dd}t|dd}|j}|j}|j} |j } |j } |T|R||tjd|dy||ks||kDr(tjd|d|d|d |d|d  y| E|tjd|d y|| kr"tjd|d |d | d y| | ttfD]} t| syd} t|ts,t|tstjdd| y|j| k7s|j | k7r._activation_post_process_satisfies_dtype_config_constraints\sw >>UV+t< +t<  %6 2HH2HH2HH$:$L$L!)?)V)V&  (->-J$ (= |n,`ah`ij!22mFW6W |n-((5bH..?-@CTBUV 'y*   ($ |n,J7)T00 |nF=/B66G5H U\T]_ & 1.:&<=T$U  !'+;< Q  ')= !8:RS 77>is>BRT";;&&*HH +HNN+;;MhNaNaMbc33L2MUSqRrs 'yN+;= rMT activationrU) rrr r rDrHrrrarUr)rrDrEr`activation_post_process_ctrrGsatisfies_constraintsrAs` rN+_qconfig_satisfies_dtype_config_constraintsrdIs&N!&|5E'E!FN 4NN  N`066>, $1rzs  "!11 7E  A3*&2. 4 HH"" HH"" HH%%  H H  H ,T , , , *4 *c *d *- 4d +; ;<- #Y->I ( x &#("(4:*>8 '+Dz811S1 HHOO$)36?B *P Pc588??23P<@t$? ?@ =59"&   #ryy. !!bii1  d^ @  CHo   8.8FJ8 49d49o %&82" = =UXX__,-== sm =  =2" K KUXX__,-KK sm K  K,   #C$89  ehhoo = = 88??=UXX__,-=  =  =(\ \ 88??\UXX__,-\  \  \~D DUXX__,-Dd^DN`=%`=F ?"<1A#AB ? ?"v! v!0v!v! v!rM