% &h6hddlZddlmZddlmZddlmZddlmZm Z m Z m Z ddl Z ddl mZddlmZddlmZmZmZdd lmZdd lmZdd lmZmZmZdd lmZmZdd l m!Z!m"Z"ddl#m$Z$ddl%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+ddl,m-Z-ddl.m/Z/m0Z0ddl1m2Z2m3Z3e*jhe5Z6dZ7eGddeZ8GddejrZ:GddejvZ<GddejvZ=GddejvZ>d Z?dCd!Z@d"e jd#eBd$e jfd%ZC dDd&ejvd'e jd(e jd)e jd*e e jd+eDd,e eDd-e eDd$e e je jffd.ZEGd/d0ejvZFGd1d2ejvZGd3ZHe&d4eHGd5d6e"ZId7ZJe&d8eHGd9d:eIZKGd;dejvZMe&d?eHGd@dAeIeZNgdBZOy)EN)Callable) dataclass)partial)ListOptionalTupleUnion)ACT2FN)Cache HybridCache StaticCache)GenerationMixin)FlashAttentionKwargs)BaseModelOutputWithPastCausalLMOutputWithPast ModelOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)add_start_docstrings%add_start_docstrings_to_model_forwardcan_return_tupleis_torchdynamo_compilingloggingreplace_return_docstrings)deprecate_kwarg) AutoModelAutoModelForCausalLM) Gemma3ConfigGemma3TextConfigr$ceZdZUdZdZeejed<dZ eejed<dZ ee e eje fed<dZeeejed<dZeeejed<dZeejed<y) Gemma3CausalLMOutputWithPasta Base class for Gemma3 causal language model (or autoregressive) outputs. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.text_config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder after projecting last hidden state. Nlosslogitspast_key_values hidden_states attentionsimage_hidden_states)__name__ __module__ __qualname____doc__r(rtorch FloatTensor__annotations__r)r*r rr r+rr,r-/var/www/pru.catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/transformers/models/gemma3/modeling_gemma3.pyr'r'8s<)-D(5$$ %,*.FHU&& '.GKOXeD):):$;U$BCDK8IJJJr6c|j|j}|d|jjzz}|j|S)NrN)rtrPrLtype_as)rErfoutputs r7rJzGemma3RMSNorm.forwardsCAGGI&3!2!2!445~~a  r6c^t|jjd|jS)Nz, eps=)tuplerLshaperjrEs r7 extra_reprzGemma3RMSNorm.extra_reprs' ))*+6$((<) r.r/r0rOrPrBrtrJr|rRrSs@r7rhrh}s&5C5e5 K!=r6rhc^eZdZddeffd Zej edZxZ S)Gemma3RotaryEmbeddingrVct|t|drG|j;|jj d|jj d|_nd|_|j |_|j |_||_ t|j |_ |j|j|\}|_ |jd|d|j|_y)N rope_scaling rope_typetypedefaultinv_freqFr?)rArBhasattrrgetrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenrVr rope_init_fnattention_scalingrCroriginal_inv_freq)rErVdevicerrFs r7rBzGemma3RotaryEmbedding.__init__s  6> *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E!((--[`J`ahhmmfk ^^ U C 5&,,.1F1L1L1NNYYZ[]^_E))UEN3C'')d444C'')d444C  5 vvAGGv$cff177f&;;;  5 5s BF%%F.rI) r.r/r0r%rBr2no_gradrrJrRrSs@r7r~r~s4///"U]]_ <