Jul 29 21:16:28.667512 ip-10-0-129-242 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 21:16:29.050999 ip-10-0-129-242 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 21:16:29.050999 ip-10-0-129-242 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 21:16:29.050999 ip-10-0-129-242 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 21:16:29.050999 ip-10-0-129-242 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 21:16:29.050999 ip-10-0-129-242 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 21:16:29.051937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.051847    2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 21:16:29.058151 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058124    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:16:29.058151 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058147    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:16:29.058151 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058152    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:16:29.058151 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058155    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:16:29.058151 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058158    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:16:29.058151 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058161    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058165    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058178    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058181    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058184    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058186    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058189    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058192    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058194    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058197    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058200    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058202    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058205    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058207    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058210    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058212    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058216    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058220    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058223    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:16:29.058412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058226    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058228    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058231    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058234    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058237    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058240    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058243    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058246    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058249    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058251    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058254    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058257    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058259    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058262    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058265    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058270    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058273    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058277    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058280    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:16:29.058858 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058283    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058286    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058289    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058292    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058295    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058298    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058301    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058304    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058307    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058310    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058312    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058315    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058318    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058321    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058324    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058327    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058329    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058332    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058334    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058338    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:16:29.059330 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058343    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058346    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058349    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058352    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058361    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058364    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058367    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058370    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058373    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058376    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058379    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058381    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058384    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058387    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058389    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058392    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058394    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058397    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058400    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058402    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:16:29.059910 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058406    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058409    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058411    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058414    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058417    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058419    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058422    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058425    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058428    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058430    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058434    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058438    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058441    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058444    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058446    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.058449    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059185    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059192    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059195    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:16:29.060458 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059198    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059201    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059204    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059207    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059210    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059213    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059215    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059218    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059220    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059223    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059226    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059228    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059231    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059234    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059237    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059240    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059243    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059245    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059248    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059251    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:16:29.060907 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059253    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059256    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059258    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059260    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059263    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059266    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059269    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059271    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059274    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059277    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059279    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059282    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059285    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059290    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059293    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059296    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059299    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059301    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059304    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:16:29.061431 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059306    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059309    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059311    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059314    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059316    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059319    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059321    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059324    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059327    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059332    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059337    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059340    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059344    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059347    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059350    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059353    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059355    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059358    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059361    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059363    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:16:29.061900 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059366    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059368    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059371    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059374    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059377    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059379    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059383    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059385    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059389    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059393    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059395    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059398    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059401    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059405    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059407    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059410    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059413    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059416    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059418    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:16:29.062406 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059421    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059423    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059426    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059430    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059432    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059435    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059437    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059440    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059443    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059445    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059448    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059450    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059453    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059455    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059458    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059461    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059463    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059465    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059843    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059849    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:16:29.062864 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059853    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059856    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059860    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059862    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059865    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059868    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059870    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059873    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059876    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059878    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059881    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059883    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059886    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059889    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059892    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059895    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059897    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059900    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059902    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059905    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:16:29.063366 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059907    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059910    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059913    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059916    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059920    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059924    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059927    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059929    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059932    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059935    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059938    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059941    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059944    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059946    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059949    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059951    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059954    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059956    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059959    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:16:29.063857 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059961    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059964    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059966    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059969    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059971    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059974    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059976    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059979    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059982    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059985    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059987    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059990    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059992    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059995    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.059997    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060000    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060003    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060005    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060008    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060011    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:16:29.064348 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060013    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060015    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060018    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060020    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060024    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060026    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060031    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060033    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060036    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060039    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060041    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060043    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060046    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060049    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060051    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060054    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060056    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060059    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060062    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060064    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:16:29.064843 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060068    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060072    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060076    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060079    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060082    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060085    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060087    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060090    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060093    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060095    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060098    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060100    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060103    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060105    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060108    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060110    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060113    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060115    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060201    2572 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060208    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 21:16:29.065390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060215    2572 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060220    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060224    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060228    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060232    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060237    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060240    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060243    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060247    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060251    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060254    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060257    2572 flags.go:64] FLAG: --cgroup-root=""
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060260    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060263    2572 flags.go:64] FLAG: --client-ca-file=""
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060268    2572 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060271    2572 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060276    2572 flags.go:64] FLAG: --cluster-domain=""
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060279    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060282    2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060285    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060288    2572 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060292    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060295    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 21:16:29.065878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060298    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060302    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060305    2572 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060308    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060311    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060314    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060317    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060321    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060324    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060327    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060330    2572 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060335    2572 flags.go:64] FLAG: --enable-server="true"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060338    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060343    2572 flags.go:64] FLAG: --event-burst="100"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060346    2572 flags.go:64] FLAG: --event-qps="50"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060349    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060352    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060355    2572 flags.go:64] FLAG: --eviction-hard=""
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060360    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060363    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060366    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060369    2572 flags.go:64] FLAG: --eviction-soft=""
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060372    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060375    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060379    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 21:16:29.066455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060381    2572 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060384    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060387    2572 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060390    2572 flags.go:64] FLAG: --feature-gates=""
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060393    2572 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060396    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060399    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060402    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060405    2572 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060408    2572 flags.go:64] FLAG: --help="false"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060411    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060414    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060417    2572 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060420    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060423    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060426    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060429    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060432    2572 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060436    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060438    2572 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060441    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060444    2572 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060447    2572 flags.go:64] FLAG: --kube-reserved=""
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060450    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 21:16:29.067090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060453    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060456    2572 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060459    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060462    2572 flags.go:64] FLAG: --lock-file=""
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060465    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060469    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060472    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060477    2572 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060481    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060484    2572 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060486    2572 flags.go:64] FLAG: --logging-format="text"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060489    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060492    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060495    2572 flags.go:64] FLAG: --manifest-url=""
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060498    2572 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060502    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060505    2572 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060509    2572 flags.go:64] FLAG: --max-pods="110"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060512    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060514    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060517    2572 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060520    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060524    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060527    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060530    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 21:16:29.067725 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060537    2572 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060541    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060544    2572 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060547    2572 flags.go:64] FLAG: --pod-cidr=""
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060550    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060555    2572 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060558    2572 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060561    2572 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060564    2572 flags.go:64] FLAG: --port="10250"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060567    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060570    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f4defa66e275065a"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060573    2572 flags.go:64] FLAG: --qos-reserved=""
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060576    2572 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060579    2572 flags.go:64] FLAG: --register-node="true"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060582    2572 flags.go:64] FLAG: --register-with-taints=""
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060586    2572 flags.go:64] FLAG: --registry-burst="10"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060597    2572 flags.go:64] FLAG: --registry-qps="5"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060600    2572 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060603    2572 flags.go:64] FLAG: --reserved-memory=""
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060611    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060614    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060617    2572 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060620    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060623    2572 flags.go:64] FLAG: --runonce="false"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060625    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 21:16:29.068357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060629    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060632    2572 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060634    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060638    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060641    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060644    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060647    2572 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060649    2572 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060653    2572 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060656    2572 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060659    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060662    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060665    2572 flags.go:64] FLAG: --system-cgroups=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060668    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060673    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060676    2572 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060679    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060683    2572 flags.go:64] FLAG: --tls-min-version=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060686    2572 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060689    2572 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060694    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060696    2572 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060700    2572 flags.go:64] FLAG: --v="2"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060704    2572 flags.go:64] FLAG: --version="false"
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060708    2572 flags.go:64] FLAG: --vmodule=""
Jul 29 21:16:29.069002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060712    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.060716    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060828    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060832    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060835    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060838    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060841    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060844    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060846    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060849    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060851    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060854    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060857    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060859    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060862    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060866    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060869    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060871    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060874    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060877    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:16:29.069629 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060879    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060881    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060885    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060888    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060892    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060894    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060897    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060902    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060904    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060908    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060912    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060916    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060919    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060922    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060925    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060927    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060930    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060933    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060935    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:16:29.070111 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060938    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060940    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060943    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060945    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060948    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060951    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060953    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060956    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060960    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060963    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060965    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060968    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060972    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060975    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060978    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060981    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060984    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060987    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060989    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:16:29.070594 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060992    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.060997    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061000    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061003    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061006    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061008    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061011    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061014    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061016    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061019    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061022    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061024    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061027    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061030    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061032    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061035    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061038    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061040    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061043    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061046    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:16:29.071064 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061048    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061053    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061055    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061058    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061060    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061063    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061065    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061068    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061071    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061073    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061076    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061079    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061081    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061086    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061088    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061091    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061093    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061096    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061098    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061101    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:16:29.071562 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061104    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061107    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.061110    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.061642    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.068814    2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.068830    2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.068852    2572 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.068858    2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068914    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068919    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068922    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068926    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068929    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068931    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068935    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:16:29.072054 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068938    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068941    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068943    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068946    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068949    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068952    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068955    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068957    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068960    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068962    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068965    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068968    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068971    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068973    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068976    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068978    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068981    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068983    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068985    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068988    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:16:29.072462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068991    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068993    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.068997    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069000    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069003    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069006    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069009    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069011    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069014    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069017    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069020    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069024    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069028    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069031    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069034    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069036    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069039    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069042    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069044    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069047    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:16:29.072933 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069049    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069054    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069057    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069060    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069063    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069066    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069069    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069072    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069075    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069078    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069080    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069083    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069087    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069090    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069095    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069099    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069102    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069105    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069107    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:16:29.073539 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069110    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069113    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069115    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069118    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069121    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069123    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069126    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069128    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069131    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069133    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069136    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069138    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069141    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069143    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069146    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069148    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069151    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069154    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069156    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069159    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:16:29.073993 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069161    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069163    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069181    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069186    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069191    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069195    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069198    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069201    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069203    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069207    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069209    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069212    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069215    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.069221    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069341    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 21:16:29.074565 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069347    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069350    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069354    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069357    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069360    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069362    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069365    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069368    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069371    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069374    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069377    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069379    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069382    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069385    2572 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069388    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069390    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069393    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069395    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069398    2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069400    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 21:16:29.074950 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069403    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069405    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069408    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069410    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069413    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069417    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069419    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069422    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069425    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069427    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069430    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069433    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069435    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069438    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069440    2572 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069442    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069445    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069448    2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069450    2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069452    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 21:16:29.075452 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069455    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069457    2572 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069460    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069463    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069465    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069468    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069470    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069473    2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069475    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069479    2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069483    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069486    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069489    2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069491    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069495    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069499    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069501    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069504    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 21:16:29.075928 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069507    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069511    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069514    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069517    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069520    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069523    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069526    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069528    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069531    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069533    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069535    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069538    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069541    2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069544    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069546    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069548    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069551    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069555    2572 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069559    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069563    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 21:16:29.076393 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069567    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069571    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069574    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069579    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069583    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069586    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069589    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069592    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069594    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069597    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069600    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069603    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069606    2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069609    2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069611    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069615    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069618    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069620    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069623    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 21:16:29.076874 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:29.069626    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.069631    2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.069756    2572 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.073101    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.074377    2572 server.go:1019] "Starting client certificate rotation"
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.074474    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.074508    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 21:16:29.077466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.077013    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 21:16:29.078672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.078651    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 21:16:29.089311 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.089287    2572 log.go:25] "Validated CRI v1 runtime API"
Jul 29 21:16:29.091434 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.091419    2572 log.go:25] "Validated CRI v1 image API"
Jul 29 21:16:29.092216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.092193    2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 21:16:29.095320 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.095299    2572 fs.go:135] Filesystem UUIDs: map[061d9de9-badb-4305-9b27-82d91e32f944:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 becbbf68-4d30-48cd-a37e-da999d9d72f7:/dev/nvme0n1p4]
Jul 29 21:16:29.095403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.095319    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 21:16:29.101209 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.101186    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 21:16:29.101454 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.101335    2572 manager.go:216] Machine: {Timestamp:2026-07-29 21:16:29.099331808 +0000 UTC m=+0.316109262 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104475 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e06b02fa30fb1c9302126348c268b SystemUUID:ec2e06b0-2fa3-0fb1-c930-2126348c268b BootID:237da93d-3488-4914-af35-a9da2e5c637a Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:42:d4:8d:fa:b5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:42:d4:8d:fa:b5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:8b:a6:9f:68:0c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 21:16:29.101454 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.101454    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 21:16:29.101568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.101555    2572 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 21:16:29.104404 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.104373    2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 21:16:29.104609 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.104408    2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-242.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 21:16:29.105622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.105610    2572 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 21:16:29.105660 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.105625    2572 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 21:16:29.105660 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.105637    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 21:16:29.106191 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.106178    2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 21:16:29.107037 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.107025    2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 21:16:29.108918 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.108904    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 21:16:29.109048 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109038    2572 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 21:16:29.109095 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109070    2572 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 21:16:29.109095 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109080    2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 21:16:29.109095 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109094    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 21:16:29.109222 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109103    2572 kubelet.go:398] "Adding apiserver pod source"
Jul 29 21:16:29.109222 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109113    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 21:16:29.109515 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109502    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 21:16:29.109561 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.109521    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 21:16:29.112553 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.112529    2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 21:16:29.114260 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.114246    2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 21:16:29.114327 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.114276    2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 21:16:29.115959 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115946    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 21:16:29.116000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115963    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 21:16:29.116000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115969    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 21:16:29.116000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115975    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 21:16:29.116000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115985    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 21:16:29.116000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115991    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 21:16:29.116000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.115999    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 21:16:29.116153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116004    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 21:16:29.116153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116012    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 21:16:29.116153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116018    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 21:16:29.116153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116026    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 21:16:29.116153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116031    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 21:16:29.116772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116762    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 21:16:29.116772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.116771    2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 21:16:29.119374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.119353    2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-242.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 21:16:29.119374 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.119358    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 21:16:29.119491 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.119385    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 21:16:29.120314 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.120299    2572 server.go:1268] "Started kubelet"
Jul 29 21:16:29.120425 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.120378    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 21:16:29.120492 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.120430    2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 21:16:29.120492 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.120481    2572 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 21:16:29.121354 ip-10-0-129-242 systemd[1]: Started Kubernetes Kubelet.
Jul 29 21:16:29.123673 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.123643    2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 21:16:29.124872 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.124731    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h6psj"
Jul 29 21:16:29.126752 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.125820    2572 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 21:16:29.128681 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.128660    2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 21:16:29.130816 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.129807    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-242.ec2.internal.18c6df44dac7951f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-242.ec2.internal,UID:ip-10-0-129-242.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-242.ec2.internal,},FirstTimestamp:2026-07-29 21:16:29.120263455 +0000 UTC m=+0.337040913,LastTimestamp:2026-07-29 21:16:29.120263455 +0000 UTC m=+0.337040913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-242.ec2.internal,}"
Jul 29 21:16:29.131478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.131459    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 21:16:29.131569 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.131475    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 21:16:29.131569 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.131560    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.132219 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132197    2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 21:16:29.132219 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132206    2572 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 21:16:29.132361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132230    2572 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 21:16:29.132361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132346    2572 factory.go:55] Registering systemd factory
Jul 29 21:16:29.132361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132361    2572 factory.go:223] Registration of the systemd container factory successfully
Jul 29 21:16:29.132488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132361    2572 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 21:16:29.132488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132414    2572 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 21:16:29.132588 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132567    2572 factory.go:153] Registering CRI-O factory
Jul 29 21:16:29.132588 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132581    2572 factory.go:223] Registration of the crio container factory successfully
Jul 29 21:16:29.132669 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132653    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 21:16:29.132716 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132677    2572 factory.go:103] Registering Raw factory
Jul 29 21:16:29.132716 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.132691    2572 manager.go:1196] Started watching for new ooms in manager
Jul 29 21:16:29.133019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.133006    2572 manager.go:349] Starting recovery of all containers
Jul 29 21:16:29.134971 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.134946    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h6psj"
Jul 29 21:16:29.143228 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.139795    2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 21:16:29.143228 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.140365    2572 manager.go:354] Recovery completed
Jul 29 21:16:29.143228 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.140809    2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 21:16:29.147763 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.147746    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.149669 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.149657    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.149758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.149679    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.149758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.149691    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.150317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.150295    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 21:16:29.150317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.150316    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 21:16:29.150522 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.150335    2572 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 21:16:29.150522 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.150343    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.151124 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.151103    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.151234 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.151139    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.151234 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.151154    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.151980 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.151963    2572 policy_none.go:49] "None policy: Start"
Jul 29 21:16:29.152037 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.151985    2572 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 21:16:29.152037 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.152000    2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 21:16:29.153138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.153125    2572 policy_none.go:47] "Start"
Jul 29 21:16:29.179880 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.179731    2572 manager.go:337] "Starting Device Plugin manager"
Jul 29 21:16:29.179964 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.179920    2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 21:16:29.179964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.179936    2572 server.go:87] "Starting device plugin registration server"
Jul 29 21:16:29.180235 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.180220    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 21:16:29.180290 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.180238    2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 21:16:29.180332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.180322    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 21:16:29.180648 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.180394    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 21:16:29.180648 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.180407    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 21:16:29.180944 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.180925    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 21:16:29.180993 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.180978    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.197382 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.197362    2572 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2603 into the system container \"/system.slice\": <nil>"
Jul 29 21:16:29.280669 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.280632    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.282093 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.282071    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.282784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.282111    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.282784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.282127    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.282784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.282162    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.290468 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.290447    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.290536 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.290480    2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-242.ec2.internal\": node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.295798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.295766    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 21:16:29.297254 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.297226    2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 21:16:29.297373 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.297268    2572 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 21:16:29.297373 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.297295    2572 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 21:16:29.297373 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.297334    2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 21:16:29.299711 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.299691    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:16:29.319148 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.319075    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.397633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.397604    2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"]
Jul 29 21:16:29.397739 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.397712    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.399378 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.399361    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.399461 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.399399    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.399461 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.399417    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.400729 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.400716    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.400862 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.400847    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.400908 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.400877    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.401452 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.401439    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.401498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.401455    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.401498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.401468    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.401498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.401476    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.401498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.401481    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.401498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.401487    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.402917 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.402903    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.402964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.402928    2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 21:16:29.403598 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.403584    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 21:16:29.403658 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.403607    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 21:16:29.403658 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.403616    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeHasSufficientPID"
Jul 29 21:16:29.419854 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.419838    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.434638 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.434616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4b1019ff0d03c1e3fd7694711ab7fc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal\" (UID: \"d4b1019ff0d03c1e3fd7694711ab7fc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.434707 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.434663    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4b1019ff0d03c1e3fd7694711ab7fc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal\" (UID: \"d4b1019ff0d03c1e3fd7694711ab7fc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.434707 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.434693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bff06bf76c4dee0b9650b26a17b25f48-config\") pod \"kube-apiserver-proxy-ip-10-0-129-242.ec2.internal\" (UID: \"bff06bf76c4dee0b9650b26a17b25f48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.436024 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.436010    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-242.ec2.internal\" not found" node="ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.440766 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.440752    2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-242.ec2.internal\" not found" node="ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.520067 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.520039    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.535053 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.535024    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4b1019ff0d03c1e3fd7694711ab7fc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal\" (UID: \"d4b1019ff0d03c1e3fd7694711ab7fc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.535118 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.535060    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4b1019ff0d03c1e3fd7694711ab7fc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal\" (UID: \"d4b1019ff0d03c1e3fd7694711ab7fc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.535151 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.535113    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bff06bf76c4dee0b9650b26a17b25f48-config\") pod \"kube-apiserver-proxy-ip-10-0-129-242.ec2.internal\" (UID: \"bff06bf76c4dee0b9650b26a17b25f48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.535151 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.535136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d4b1019ff0d03c1e3fd7694711ab7fc6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal\" (UID: \"d4b1019ff0d03c1e3fd7694711ab7fc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.535242 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.535146    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d4b1019ff0d03c1e3fd7694711ab7fc6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal\" (UID: \"d4b1019ff0d03c1e3fd7694711ab7fc6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.535242 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.535159    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bff06bf76c4dee0b9650b26a17b25f48-config\") pod \"kube-apiserver-proxy-ip-10-0-129-242.ec2.internal\" (UID: \"bff06bf76c4dee0b9650b26a17b25f48\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.620529 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.620453    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.720952 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.720923    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.738423 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.738399    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.744266 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:29.744245    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:29.821460 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.821432    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:29.921956 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:29.921893    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:30.022443 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.022417    2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-242.ec2.internal\" not found"
Jul 29 21:16:30.042637 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.042618    2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:16:30.074026 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.074007    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 21:16:30.074536 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.074183    2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 21:16:30.111131 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.111111    2572 apiserver.go:52] "Watching apiserver"
Jul 29 21:16:30.120659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.120632    2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 21:16:30.120992 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.120973    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-f7ghg","kube-system/konnectivity-agent-5vkp2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms","openshift-dns/node-resolver-4j287","openshift-multus/multus-additional-cni-plugins-5zgsh","openshift-multus/multus-fvq2g","openshift-network-diagnostics/network-check-target-jqk4m","openshift-network-operator/iptables-alerter-f8kpz","openshift-cluster-node-tuning-operator/tuned-qwjd4","openshift-image-registry/node-ca-qsc5x","openshift-multus/network-metrics-daemon-8g669","openshift-ovn-kubernetes/ovnkube-node-9nwdl"]
Jul 29 21:16:30.122552 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.122535    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.122673 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.122633    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:30.123930 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.123913    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.125259 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.125232    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.126528 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.126501    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hk8sh\""
Jul 29 21:16:30.126528 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.126517    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 21:16:30.126681 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.126570    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 21:16:30.126931 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.126914    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.127491 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.127469    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 21:16:30.127715 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.127695    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9dlx5\""
Jul 29 21:16:30.127833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.127816    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.128025 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.128008    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.129036 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.128968    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.129281 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.129258    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.129505 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.129481    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.129618 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.129543    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-452fh\""
Jul 29 21:16:30.130360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.130339    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.131708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.131687    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:30.131811 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.131715    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:30.131811 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.131761    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:30.131811 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.131690    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 21:16:30.132014 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.131968    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 21:16:30.132141 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132125    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.132191 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132164    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 21:16:30.132424 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132413    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 21:16:30.132869 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132682    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lt4vl\""
Jul 29 21:16:30.132869 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132761    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.132869 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132815    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 21:16:30.133043 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.132992    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tdvf6\""
Jul 29 21:16:30.133262 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.133237    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.134575 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.134558    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.135492 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.135472    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x6v47\""
Jul 29 21:16:30.135684 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.135663    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.135845 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.135832    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.135956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.135935    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 21:16:30.136182 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.136164    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.136597 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.136578    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.136876 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.136859    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.136964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.136915    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-l4np9\""
Jul 29 21:16:30.137460 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137442    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.137538 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.137505    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:30.137878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137850    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-netns\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.137964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137890    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-kubelet\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.137964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137919    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-host-slash\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.137964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137942    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-cnibin\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137965    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-os-release\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.137990    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xv62\" (UniqueName: \"kubernetes.io/projected/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-kube-api-access-7xv62\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.138092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138027    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-registration-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.138092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138048    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138067    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljf4h\" (UniqueName: \"kubernetes.io/projected/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-kube-api-access-ljf4h\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138083    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-cni-multus\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138098    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-kubelet-config\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138116    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-sys-fs\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138135    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138152    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138168    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87efc84b-d7a4-489a-a301-4af34121c04b-cni-binary-copy\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138202    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdnt6\" (UniqueName: \"kubernetes.io/projected/749a806a-7d33-41cd-9f4d-3c262c8d7f80-kube-api-access-xdnt6\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138225    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-os-release\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-cni-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-socket-dir-parent\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138298    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-hostroot\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138321    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-conf-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138348    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgqk5\" (UniqueName: \"kubernetes.io/projected/87efc84b-d7a4-489a-a301-4af34121c04b-kube-api-access-rgqk5\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138365    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-socket-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138393    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cnibin\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-multus-certs\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-iptables-alerter-script\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138468    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-dbus\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138500    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c5aa1de-e609-490d-80b4-0104d630b0f4-konnectivity-ca\") pod \"konnectivity-agent-5vkp2\" (UID: \"5c5aa1de-e609-490d-80b4-0104d630b0f4\") " pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138524    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138538    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0679383-8d53-4278-958c-b66a18435ce0-tmp-dir\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138570    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-etc-selinux\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138594    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87efc84b-d7a4-489a-a301-4af34121c04b-multus-daemon-config\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138619    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-etc-kubernetes\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138643    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c5aa1de-e609-490d-80b4-0104d630b0f4-agent-certs\") pod \"konnectivity-agent-5vkp2\" (UID: \"5c5aa1de-e609-490d-80b4-0104d630b0f4\") " pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138666    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0679383-8d53-4278-958c-b66a18435ce0-hosts-file\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138690    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-device-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138713    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-system-cni-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.138913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138733    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2l4n6\""
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138757    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-cni-bin\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138782    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138799    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22kh4\" (UniqueName: \"kubernetes.io/projected/d0679383-8d53-4278-958c-b66a18435ce0-kube-api-access-22kh4\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138834    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138858    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-kubelet-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138884    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-system-cni-dir\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138908    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-k8s-cni-cncf-io\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.138990    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.139060    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 21:11:29 +0000 UTC" deadline="2028-03-05 22:21:49.730426543 +0000 UTC"
Jul 29 21:16:30.139584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.139101    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14041h5m19.591329283s"
Jul 29 21:16:30.141325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.141308    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 21:16:30.141413 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.141397    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 21:16:30.141533 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.141517    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8h4d5\""
Jul 29 21:16:30.141622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.141607    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 21:16:30.141952 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.141937    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 21:16:30.142094 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.142081    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 21:16:30.142173 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.142159    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 21:16:30.145128 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.145112    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal"]
Jul 29 21:16:30.145172 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.145130    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 21:16:30.145214 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.145203    2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"
Jul 29 21:16:30.153583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.153564    2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 21:16:30.156461 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.156440    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 21:16:30.156637 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.156621    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal"]
Jul 29 21:16:30.176726 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.176677    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-7gwqz"
Jul 29 21:16:30.190448 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.190426    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7gwqz"
Jul 29 21:16:30.233863 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.233838    2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 21:16:30.239267 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239235    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-ovn\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.239390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239270    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-cnibin\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.239390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239292    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-modprobe-d\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.239390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239309    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysconfig\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.239390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239325    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysctl-d\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.239390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239353    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239395    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovnkube-script-lib\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239429    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239433    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-cnibin\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239457    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljf4h\" (UniqueName: \"kubernetes.io/projected/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-kube-api-access-ljf4h\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239484    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-cni-multus\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239512    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-kubelet-config\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239537    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-sys\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.239589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239565    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-log-socket\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239593    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-sys-fs\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239618    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c54c0b98-fe67-4c62-9663-7a21dccf869b-host\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239654    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-run\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-kubelet-config\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239672    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-cni-multus\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239684    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-socket-dir-parent\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239681    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-sys-fs\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239738    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-conf-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239769    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgqk5\" (UniqueName: \"kubernetes.io/projected/87efc84b-d7a4-489a-a301-4af34121c04b-kube-api-access-rgqk5\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239796    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-iptables-alerter-script\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239846    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-socket-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239882    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-conf-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239884    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cnibin\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239920    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cnibin\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239923    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87efc84b-d7a4-489a-a301-4af34121c04b-multus-daemon-config\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239887    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-socket-dir-parent\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239952    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-socket-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.239990    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ln7\" (UniqueName: \"kubernetes.io/projected/c54c0b98-fe67-4c62-9663-7a21dccf869b-kube-api-access-w9ln7\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240026    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-host\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240055    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-run-netns\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-system-cni-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240107    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-cni-bin\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240135    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240161    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-cni-bin\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240163    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22kh4\" (UniqueName: \"kubernetes.io/projected/d0679383-8d53-4278-958c-b66a18435ce0-kube-api-access-22kh4\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240217    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-system-cni-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240227    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-var-lib-kubelet\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240258    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-kubelet-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-system-cni-dir\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.240290    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240310    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysctl-conf\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-tuned\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240352    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-kubelet-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.240788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240360    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzwvv\" (UniqueName: \"kubernetes.io/projected/a9e2641f-73be-4869-bad7-41bdfdde4d93-kube-api-access-gzwvv\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.240378    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:16:30.7403555 +0000 UTC m=+1.957132974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240409    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-system-cni-dir\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240431    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-cni-bin\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240462    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-os-release\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240488    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-iptables-alerter-script\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240510    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xv62\" (UniqueName: \"kubernetes.io/projected/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-kube-api-access-7xv62\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240510    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87efc84b-d7a4-489a-a301-4af34121c04b-multus-daemon-config\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240537    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-registration-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240553    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-os-release\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240581    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-kubernetes\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240607    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9e2641f-73be-4869-bad7-41bdfdde4d93-tmp\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240605    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-registration-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240637    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-var-lib-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240638    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240685    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-node-log\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.241679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240713    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-cni-netd\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chm2h\" (UniqueName: \"kubernetes.io/projected/218e972a-89b8-4c61-82e0-d58efb52e1f9-kube-api-access-chm2h\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240756    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240774    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240790    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87efc84b-d7a4-489a-a301-4af34121c04b-cni-binary-copy\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240810    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-systemd\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240849    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240892    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovn-node-metrics-cert\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240922    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g5t7\" (UniqueName: \"kubernetes.io/projected/46824038-d39b-48cd-9bd4-32177f2500f2-kube-api-access-7g5t7\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.240955    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdnt6\" (UniqueName: \"kubernetes.io/projected/749a806a-7d33-41cd-9f4d-3c262c8d7f80-kube-api-access-xdnt6\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241013    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-os-release\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241045    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-cni-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-hostroot\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241151    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-os-release\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241211    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-dbus\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241228    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-multus-cni-dir\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241242    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c5aa1de-e609-490d-80b4-0104d630b0f4-konnectivity-ca\") pod \"konnectivity-agent-5vkp2\" (UID: \"5c5aa1de-e609-490d-80b4-0104d630b0f4\") " pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.242483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241300    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241325    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87efc84b-d7a4-489a-a301-4af34121c04b-cni-binary-copy\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241336    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241329    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241355    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241380    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-multus-certs\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241408    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0679383-8d53-4278-958c-b66a18435ce0-tmp-dir\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241420    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-hostroot\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241435    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241460    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241476    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241485    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovnkube-config\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241481    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-dbus\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241521    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-env-overrides\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241551    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-multus-certs\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241583    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-etc-selinux\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241608    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-etc-kubernetes\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.243374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241633    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c5aa1de-e609-490d-80b4-0104d630b0f4-agent-certs\") pod \"konnectivity-agent-5vkp2\" (UID: \"5c5aa1de-e609-490d-80b4-0104d630b0f4\") " pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241669    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0679383-8d53-4278-958c-b66a18435ce0-hosts-file\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241700    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-kubelet\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241701    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-etc-selinux\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241726    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0679383-8d53-4278-958c-b66a18435ce0-tmp-dir\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241795    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-etc-kubernetes\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241806    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c5aa1de-e609-490d-80b4-0104d630b0f4-konnectivity-ca\") pod \"konnectivity-agent-5vkp2\" (UID: \"5c5aa1de-e609-490d-80b4-0104d630b0f4\") " pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241828    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-slash\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241815    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0679383-8d53-4278-958c-b66a18435ce0-hosts-file\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241850    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-device-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c54c0b98-fe67-4c62-9663-7a21dccf869b-serviceca\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241894    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749a806a-7d33-41cd-9f4d-3c262c8d7f80-device-dir\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241907    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-lib-modules\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241931    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-k8s-cni-cncf-io\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241976    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-netns\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242008    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-kubelet\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.241990    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242026    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-netns\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.244293 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242050    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-var-lib-kubelet\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.245469 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242071    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-host-slash\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.245469 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242098    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87efc84b-d7a4-489a-a301-4af34121c04b-host-run-k8s-cni-cncf-io\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.245469 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242106    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-systemd-units\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.245469 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242146    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-host-slash\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.245469 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242146    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-systemd\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.245469 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.242185    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-etc-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.247064 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.246673    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c5aa1de-e609-490d-80b4-0104d630b0f4-agent-certs\") pod \"konnectivity-agent-5vkp2\" (UID: \"5c5aa1de-e609-490d-80b4-0104d630b0f4\") " pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.247991 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.247964    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:16:30.248104 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.247996    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:16:30.248104 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.248009    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:30.248104 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.248070    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:30.748049783 +0000 UTC m=+1.964827240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:30.249832 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.249807    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljf4h\" (UniqueName: \"kubernetes.io/projected/ca902c68-ff65-4b9b-8b08-dcb3e36e1d97-kube-api-access-ljf4h\") pod \"multus-additional-cni-plugins-5zgsh\" (UID: \"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97\") " pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.250341 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.250316    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xv62\" (UniqueName: \"kubernetes.io/projected/d81690a6-3ab0-437d-83ce-1ac0e92c7db9-kube-api-access-7xv62\") pod \"iptables-alerter-f8kpz\" (UID: \"d81690a6-3ab0-437d-83ce-1ac0e92c7db9\") " pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.250341 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.250328    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22kh4\" (UniqueName: \"kubernetes.io/projected/d0679383-8d53-4278-958c-b66a18435ce0-kube-api-access-22kh4\") pod \"node-resolver-4j287\" (UID: \"d0679383-8d53-4278-958c-b66a18435ce0\") " pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.250985 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.250959    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgqk5\" (UniqueName: \"kubernetes.io/projected/87efc84b-d7a4-489a-a301-4af34121c04b-kube-api-access-rgqk5\") pod \"multus-fvq2g\" (UID: \"87efc84b-d7a4-489a-a301-4af34121c04b\") " pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.251259 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.251212    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdnt6\" (UniqueName: \"kubernetes.io/projected/749a806a-7d33-41cd-9f4d-3c262c8d7f80-kube-api-access-xdnt6\") pod \"aws-ebs-csi-driver-node-87xms\" (UID: \"749a806a-7d33-41cd-9f4d-3c262c8d7f80\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.338298 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.338230    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b1019ff0d03c1e3fd7694711ab7fc6.slice/crio-a92fea0a9bae1b9cb2724c5e3eb9c8a30a36bc833372de076891849a72ebcb5d WatchSource:0}: Error finding container a92fea0a9bae1b9cb2724c5e3eb9c8a30a36bc833372de076891849a72ebcb5d: Status 404 returned error can't find the container with id a92fea0a9bae1b9cb2724c5e3eb9c8a30a36bc833372de076891849a72ebcb5d
Jul 29 21:16:30.338989 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.338958    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff06bf76c4dee0b9650b26a17b25f48.slice/crio-06f6e9bb07aa10ccc6fcd7bcf3f4389dc1a5401ef7c6d1e2d335258bd4f91c20 WatchSource:0}: Error finding container 06f6e9bb07aa10ccc6fcd7bcf3f4389dc1a5401ef7c6d1e2d335258bd4f91c20: Status 404 returned error can't find the container with id 06f6e9bb07aa10ccc6fcd7bcf3f4389dc1a5401ef7c6d1e2d335258bd4f91c20
Jul 29 21:16:30.342677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342652    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-cni-netd\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.342784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chm2h\" (UniqueName: \"kubernetes.io/projected/218e972a-89b8-4c61-82e0-d58efb52e1f9-kube-api-access-chm2h\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.342784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342706    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-systemd\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.342784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342726    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342787    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342794    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-cni-netd\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342797    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-systemd\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342826    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovn-node-metrics-cert\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g5t7\" (UniqueName: \"kubernetes.io/projected/46824038-d39b-48cd-9bd4-32177f2500f2-kube-api-access-7g5t7\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342926    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.342956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342954    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.342982    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovnkube-config\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343018    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-env-overrides\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343051    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-kubelet\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.343059    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343064    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-run-ovn-kubernetes\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343077    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-slash\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.343118    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:30.843098507 +0000 UTC m=+2.059875952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343125    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-slash\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343138    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c54c0b98-fe67-4c62-9663-7a21dccf869b-serviceca\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343147    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-kubelet\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343168    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-lib-modules\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343276    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-systemd-units\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343304    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-systemd\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.343332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343331    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-etc-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343359    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-ovn\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343389    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-modprobe-d\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343400    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-lib-modules\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343416    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysconfig\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343417    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-systemd\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343451    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-systemd-units\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysctl-d\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343475    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-etc-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343492    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343511    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysconfig\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343525    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovnkube-script-lib\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343555    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-sys\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343495    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-run-ovn\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343562    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343584    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysctl-d\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343584    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-modprobe-d\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343598    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-log-socket\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.344193 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343613    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-sys\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343648    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c54c0b98-fe67-4c62-9663-7a21dccf869b-host\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343662    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-log-socket\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343679    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-run\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343700    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c54c0b98-fe67-4c62-9663-7a21dccf869b-host\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ln7\" (UniqueName: \"kubernetes.io/projected/c54c0b98-fe67-4c62-9663-7a21dccf869b-kube-api-access-w9ln7\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343742    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-host\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-run-netns\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343783    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-run\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343806    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c54c0b98-fe67-4c62-9663-7a21dccf869b-serviceca\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343823    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-var-lib-kubelet\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343824    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovnkube-config\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343833    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-run-netns\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343873    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-host\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343878    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-var-lib-kubelet\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343907    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysctl-conf\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-tuned\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343938    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-env-overrides\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343959    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzwvv\" (UniqueName: \"kubernetes.io/projected/a9e2641f-73be-4869-bad7-41bdfdde4d93-kube-api-access-gzwvv\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.343989    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-cni-bin\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344020    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-kubernetes\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9e2641f-73be-4869-bad7-41bdfdde4d93-tmp\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344069    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-var-lib-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344078    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-sysctl-conf\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344093    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-node-log\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344080    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-host-cni-bin\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344112    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovnkube-script-lib\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344169    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-node-log\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344176    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/218e972a-89b8-4c61-82e0-d58efb52e1f9-var-lib-openvswitch\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.345774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.344263    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-kubernetes\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.346342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.345854    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/218e972a-89b8-4c61-82e0-d58efb52e1f9-ovn-node-metrics-cert\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.346342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.346139    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a9e2641f-73be-4869-bad7-41bdfdde4d93-etc-tuned\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.346641 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.346535    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:16:30.346680 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.346661    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9e2641f-73be-4869-bad7-41bdfdde4d93-tmp\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.353120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.353093    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ln7\" (UniqueName: \"kubernetes.io/projected/c54c0b98-fe67-4c62-9663-7a21dccf869b-kube-api-access-w9ln7\") pod \"node-ca-qsc5x\" (UID: \"c54c0b98-fe67-4c62-9663-7a21dccf869b\") " pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.353844 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.353822    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g5t7\" (UniqueName: \"kubernetes.io/projected/46824038-d39b-48cd-9bd4-32177f2500f2-kube-api-access-7g5t7\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.354075 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.354059    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chm2h\" (UniqueName: \"kubernetes.io/projected/218e972a-89b8-4c61-82e0-d58efb52e1f9-kube-api-access-chm2h\") pod \"ovnkube-node-9nwdl\" (UID: \"218e972a-89b8-4c61-82e0-d58efb52e1f9\") " pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.354611 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.354596    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzwvv\" (UniqueName: \"kubernetes.io/projected/a9e2641f-73be-4869-bad7-41bdfdde4d93-kube-api-access-gzwvv\") pod \"tuned-qwjd4\" (UID: \"a9e2641f-73be-4869-bad7-41bdfdde4d93\") " pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.438533 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.438512    2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:16:30.450429 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.450405    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:30.458153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.458136    2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:16:30.459377 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.459351    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c5aa1de_e609_490d_80b4_0104d630b0f4.slice/crio-0246a891b467a7f0b749b0f71552b135ae1503a2147200aa4265206614d338f4 WatchSource:0}: Error finding container 0246a891b467a7f0b749b0f71552b135ae1503a2147200aa4265206614d338f4: Status 404 returned error can't find the container with id 0246a891b467a7f0b749b0f71552b135ae1503a2147200aa4265206614d338f4
Jul 29 21:16:30.466504 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.466484    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms"
Jul 29 21:16:30.474473 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.474450    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749a806a_7d33_41cd_9f4d_3c262c8d7f80.slice/crio-e88508821c4528936001425f5c9a8a6e7bc77633065a16ea9163952086c13fbe WatchSource:0}: Error finding container e88508821c4528936001425f5c9a8a6e7bc77633065a16ea9163952086c13fbe: Status 404 returned error can't find the container with id e88508821c4528936001425f5c9a8a6e7bc77633065a16ea9163952086c13fbe
Jul 29 21:16:30.482657 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.482637    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4j287"
Jul 29 21:16:30.490247 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.490223    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0679383_8d53_4278_958c_b66a18435ce0.slice/crio-7fa13dd41f347243b842896e6b21750418b03194f76b49aa673d5d26820d381d WatchSource:0}: Error finding container 7fa13dd41f347243b842896e6b21750418b03194f76b49aa673d5d26820d381d: Status 404 returned error can't find the container with id 7fa13dd41f347243b842896e6b21750418b03194f76b49aa673d5d26820d381d
Jul 29 21:16:30.493286 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.493266    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5zgsh"
Jul 29 21:16:30.501597 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.501576    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca902c68_ff65_4b9b_8b08_dcb3e36e1d97.slice/crio-2a26b6945f18563045e0ec243db7723e46946264fad9def5306ce73e1b5da580 WatchSource:0}: Error finding container 2a26b6945f18563045e0ec243db7723e46946264fad9def5306ce73e1b5da580: Status 404 returned error can't find the container with id 2a26b6945f18563045e0ec243db7723e46946264fad9def5306ce73e1b5da580
Jul 29 21:16:30.508864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.508839    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvq2g"
Jul 29 21:16:30.513555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.513536    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8kpz"
Jul 29 21:16:30.516587 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.516560    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87efc84b_d7a4_489a_a301_4af34121c04b.slice/crio-8abd0ebde1ce4f9511f756bec27762f89fdb1f6b301100dd97dfd2decceaa10b WatchSource:0}: Error finding container 8abd0ebde1ce4f9511f756bec27762f89fdb1f6b301100dd97dfd2decceaa10b: Status 404 returned error can't find the container with id 8abd0ebde1ce4f9511f756bec27762f89fdb1f6b301100dd97dfd2decceaa10b
Jul 29 21:16:30.522426 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.522401    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81690a6_3ab0_437d_83ce_1ac0e92c7db9.slice/crio-a9fd0659844be2236a906cb01c19cbbe7f68a786d7b5094c3b7986fcb7ada782 WatchSource:0}: Error finding container a9fd0659844be2236a906cb01c19cbbe7f68a786d7b5094c3b7986fcb7ada782: Status 404 returned error can't find the container with id a9fd0659844be2236a906cb01c19cbbe7f68a786d7b5094c3b7986fcb7ada782
Jul 29 21:16:30.548672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.548646    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwjd4"
Jul 29 21:16:30.556468 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.556444    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e2641f_73be_4869_bad7_41bdfdde4d93.slice/crio-dc6fb045297c064030cb30358090e7e49bb498b2ba81986b441e34824bbe91c9 WatchSource:0}: Error finding container dc6fb045297c064030cb30358090e7e49bb498b2ba81986b441e34824bbe91c9: Status 404 returned error can't find the container with id dc6fb045297c064030cb30358090e7e49bb498b2ba81986b441e34824bbe91c9
Jul 29 21:16:30.574537 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.574516    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qsc5x"
Jul 29 21:16:30.580697 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.580677    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:30.581961 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.581936    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54c0b98_fe67_4c62_9663_7a21dccf869b.slice/crio-f03ac02282364413f02c079aa8facbcd67759e57290e40289f0b4e70a688ff75 WatchSource:0}: Error finding container f03ac02282364413f02c079aa8facbcd67759e57290e40289f0b4e70a688ff75: Status 404 returned error can't find the container with id f03ac02282364413f02c079aa8facbcd67759e57290e40289f0b4e70a688ff75
Jul 29 21:16:30.588658 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:16:30.588637    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218e972a_89b8_4c61_82e0_d58efb52e1f9.slice/crio-cd96c170d2accbba15e1157ff898bb544958d00a5230a173eea92cabd36c9240 WatchSource:0}: Error finding container cd96c170d2accbba15e1157ff898bb544958d00a5230a173eea92cabd36c9240: Status 404 returned error can't find the container with id cd96c170d2accbba15e1157ff898bb544958d00a5230a173eea92cabd36c9240
Jul 29 21:16:30.747101 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.746994    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:30.747297 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.747154    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:30.747297 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.747250    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:16:31.747229097 +0000 UTC m=+2.964006557 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:30.847993 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.847959    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:30.848213 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:30.848003    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:30.848213 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.848158    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:30.848213 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.848167    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:16:30.848213 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.848211    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:16:30.848497 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.848228    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:30.848497 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.848235    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:31.848220076 +0000 UTC m=+3.064997532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:30.848497 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:30.848283    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:31.848264319 +0000 UTC m=+3.065041775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:31.193170 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.193091    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 21:11:30 +0000 UTC" deadline="2028-03-04 07:21:40.765651664 +0000 UTC"
Jul 29 21:16:31.194092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.193168    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14002h5m9.572487864s"
Jul 29 21:16:31.301592 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.301408    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:31.301592 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.301548    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:31.326984 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.326817    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"cd96c170d2accbba15e1157ff898bb544958d00a5230a173eea92cabd36c9240"}
Jul 29 21:16:31.330854 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.330773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qsc5x" event={"ID":"c54c0b98-fe67-4c62-9663-7a21dccf869b","Type":"ContainerStarted","Data":"f03ac02282364413f02c079aa8facbcd67759e57290e40289f0b4e70a688ff75"}
Jul 29 21:16:31.352920 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.352874    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwjd4" event={"ID":"a9e2641f-73be-4869-bad7-41bdfdde4d93","Type":"ContainerStarted","Data":"dc6fb045297c064030cb30358090e7e49bb498b2ba81986b441e34824bbe91c9"}
Jul 29 21:16:31.363965 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.363931    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvq2g" event={"ID":"87efc84b-d7a4-489a-a301-4af34121c04b","Type":"ContainerStarted","Data":"8abd0ebde1ce4f9511f756bec27762f89fdb1f6b301100dd97dfd2decceaa10b"}
Jul 29 21:16:31.370226 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.370117    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerStarted","Data":"2a26b6945f18563045e0ec243db7723e46946264fad9def5306ce73e1b5da580"}
Jul 29 21:16:31.374710 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.374646    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms" event={"ID":"749a806a-7d33-41cd-9f4d-3c262c8d7f80","Type":"ContainerStarted","Data":"e88508821c4528936001425f5c9a8a6e7bc77633065a16ea9163952086c13fbe"}
Jul 29 21:16:31.387974 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.387902    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5vkp2" event={"ID":"5c5aa1de-e609-490d-80b4-0104d630b0f4","Type":"ContainerStarted","Data":"0246a891b467a7f0b749b0f71552b135ae1503a2147200aa4265206614d338f4"}
Jul 29 21:16:31.404120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.404082    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal" event={"ID":"bff06bf76c4dee0b9650b26a17b25f48","Type":"ContainerStarted","Data":"06f6e9bb07aa10ccc6fcd7bcf3f4389dc1a5401ef7c6d1e2d335258bd4f91c20"}
Jul 29 21:16:31.406612 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.406582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8kpz" event={"ID":"d81690a6-3ab0-437d-83ce-1ac0e92c7db9","Type":"ContainerStarted","Data":"a9fd0659844be2236a906cb01c19cbbe7f68a786d7b5094c3b7986fcb7ada782"}
Jul 29 21:16:31.430198 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.430139    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4j287" event={"ID":"d0679383-8d53-4278-958c-b66a18435ce0","Type":"ContainerStarted","Data":"7fa13dd41f347243b842896e6b21750418b03194f76b49aa673d5d26820d381d"}
Jul 29 21:16:31.445977 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.445893    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal" event={"ID":"d4b1019ff0d03c1e3fd7694711ab7fc6","Type":"ContainerStarted","Data":"a92fea0a9bae1b9cb2724c5e3eb9c8a30a36bc833372de076891849a72ebcb5d"}
Jul 29 21:16:31.482284 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.482218    2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 21:16:31.756089 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.755993    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:31.756311 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.756258    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:31.756387 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.756329    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:16:33.756308496 +0000 UTC m=+4.973085944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.857002    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:31.857058    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.857261    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.857333    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:33.857312235 +0000 UTC m=+5.074089683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.857774    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.857800    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.857814    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:31.857906 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:31.857862    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:33.857846509 +0000 UTC m=+5.074623967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:32.194124 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:32.193926    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 21:11:30 +0000 UTC" deadline="2028-01-24 21:11:01.650601552 +0000 UTC"
Jul 29 21:16:32.194124 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:32.193976    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13055h54m29.456632955s"
Jul 29 21:16:32.298518 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:32.298484    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:32.298715 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:32.298629    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:32.299139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:32.299113    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:32.299291 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:32.299228    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:33.299896 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:33.299856    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:33.300405 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.300015    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:33.774412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:33.773704    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:33.774412 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.773872    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:33.774412 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.773944    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:16:37.77392283 +0000 UTC m=+8.990700293 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:33.874994    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:33.875044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.875199    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.875282    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:37.875260455 +0000 UTC m=+9.092037900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.875723    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.875745    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.875759    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:33.876149 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:33.875812    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:37.875793117 +0000 UTC m=+9.092570559 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:34.305570 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:34.300633    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:34.305570 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:34.300824    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:34.305570 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:34.301361    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:34.305570 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:34.301499    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:35.300318 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:35.300282    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:35.300502 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:35.300409    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:36.299032 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:36.298359    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:36.299032 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:36.298505    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:36.299032 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:36.298883    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:36.299032 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:36.298984    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:37.298257 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:37.298084    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:37.298456 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.298262    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:37.816489 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:37.816456    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:37.816954 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.816597    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:37.816954 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.816656    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:16:45.816642634 +0000 UTC m=+17.033420093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:37.917388 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:37.917345    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:37.917571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:37.917399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:37.917571 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.917541    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:37.917685 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.917606    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:45.91758687 +0000 UTC m=+17.134364326 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:37.917685 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.917679    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:16:37.917801 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.917701    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:16:37.917801 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.917716    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:37.917801 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:37.917759    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:16:45.917744944 +0000 UTC m=+17.134522404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:38.298731 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:38.298027    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:38.298731 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:38.298390    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:38.298731 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:38.298414    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:38.298731 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:38.298533    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:39.299436 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:39.298945    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:39.299436 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:39.299074    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:40.298016 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:40.297978    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:40.298206 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:40.297985    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:40.298206 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:40.298109    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:40.298311 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:40.298213    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:41.298514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:41.298475    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:41.298882 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:41.298608    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:42.298343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:42.298311    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:42.298526 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:42.298440    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:42.298526 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:42.298491    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:42.298926 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:42.298576    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:43.298054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:43.298023    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:43.298250 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:43.298150    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:44.298501 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:44.298459    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:44.298501 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:44.298501    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:44.298914 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:44.298579    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:44.298914 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:44.298649    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:45.298622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:45.298584    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:45.299100 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.298727    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:45.878219 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:45.878160    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:45.878433 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.878349    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:45.878433 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.878422    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:01.878399893 +0000 UTC m=+33.095177356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:16:45.979153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:45.979114    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:45.979391 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:45.979180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:45.979391 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.979299    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:45.979391 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.979352    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:16:45.979391 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.979380    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:16:45.979391 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.979396    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:45.979610 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.979366    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:01.979347512 +0000 UTC m=+33.196124954 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:16:45.979610 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:45.979480    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:01.979453095 +0000 UTC m=+33.196230551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:16:46.298573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:46.298484    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:46.298573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:46.298542    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:46.299102 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:46.298652    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:46.299102 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:46.298748    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:47.298538 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:47.298501    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:47.298731 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:47.298624    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:48.298236 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:48.298200    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:48.298707 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:48.298250    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:48.298707 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:48.298351    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:48.298707 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:48.298417    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:49.298578 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:49.298539    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:49.299059 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:49.298664    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:50.298162 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:50.298130    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:50.298348 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:50.298261    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:50.298348 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:50.298301    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:50.298472 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:50.298398    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:51.298256 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.297949    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:51.298830 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:51.298346    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:51.495112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.495028    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qsc5x" event={"ID":"c54c0b98-fe67-4c62-9663-7a21dccf869b","Type":"ContainerStarted","Data":"7c1df45f9316ad2de4530a2db6f7bd99be9a6529b9b27d2a245c76919518a3dd"}
Jul 29 21:16:51.496534 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.496503    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwjd4" event={"ID":"a9e2641f-73be-4869-bad7-41bdfdde4d93","Type":"ContainerStarted","Data":"0e9cd28157053456ef989873236aee32c7e0471d20de39e2b237872cef2198e8"}
Jul 29 21:16:51.497908 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.497885    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvq2g" event={"ID":"87efc84b-d7a4-489a-a301-4af34121c04b","Type":"ContainerStarted","Data":"e7dc12ae2b1aba55ce6bdcd46648495b8d6cebd3029e2b0829c44a4ca77192b9"}
Jul 29 21:16:51.499403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.499379    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca902c68-ff65-4b9b-8b08-dcb3e36e1d97" containerID="35a6dffb6dad085c909c901078f2c223633d66c2bd28e4421f65e68ce1df8fb5" exitCode=0
Jul 29 21:16:51.499488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.499450    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerDied","Data":"35a6dffb6dad085c909c901078f2c223633d66c2bd28e4421f65e68ce1df8fb5"}
Jul 29 21:16:51.500939 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.500905    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms" event={"ID":"749a806a-7d33-41cd-9f4d-3c262c8d7f80","Type":"ContainerStarted","Data":"dfffd77fe3eda7fb06bd02f7275073ea97f0435808be596c315902fb21fa2624"}
Jul 29 21:16:51.502333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.502314    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5vkp2" event={"ID":"5c5aa1de-e609-490d-80b4-0104d630b0f4","Type":"ContainerStarted","Data":"78de2595031154331b58721dc1c45f82037a431ef6304cda160be54a8c26de95"}
Jul 29 21:16:51.503680 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.503654    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal" event={"ID":"bff06bf76c4dee0b9650b26a17b25f48","Type":"ContainerStarted","Data":"b3c41556f29c153ea57f7943a4821018d281e53209e87f8c38967dacca91b6d8"}
Jul 29 21:16:51.504993 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.504970    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4j287" event={"ID":"d0679383-8d53-4278-958c-b66a18435ce0","Type":"ContainerStarted","Data":"dbf29ecacd2f0f4ed421ded6c9473efd9daf939be3ae416be151da7b5532e4d5"}
Jul 29 21:16:51.506859 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.506833    2572 generic.go:356] "Generic (PLEG): container finished" podID="d4b1019ff0d03c1e3fd7694711ab7fc6" containerID="4c53a823868e90e77717e94c3ffd63014379a67b6b34eced8218428468812d63" exitCode=0
Jul 29 21:16:51.507011 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.506880    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal" event={"ID":"d4b1019ff0d03c1e3fd7694711ab7fc6","Type":"ContainerDied","Data":"4c53a823868e90e77717e94c3ffd63014379a67b6b34eced8218428468812d63"}
Jul 29 21:16:51.509836 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.509815    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"28967d9024af5bce9d71155f1a5e2cefdcbfce54233891424d30d6aa4bcc3262"}
Jul 29 21:16:51.509927 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.509869    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"730107cf67fe18e6d0e8f155c4447e289476f72154e90dec5d5e941360e01ab3"}
Jul 29 21:16:51.509927 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.509904    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"a46561f29b51f8882363a77f48772f816861fe6fb4f690b6c1433805d37b9595"}
Jul 29 21:16:51.510040 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.509932    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"cc02e70f2894a9114ef5ad3c869ea7adf63bbd1336e49971f57f7935274d830e"}
Jul 29 21:16:51.512361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.512310    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qsc5x" podStartSLOduration=2.289038064 podStartE2EDuration="22.512292676s" totalImagesPullingTime="20.223254612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:51.511087348 +0000 UTC m=+22.727864809" watchObservedRunningTime="2026-07-29 21:16:51.512292676 +0000 UTC m=+22.729070138"
Jul 29 21:16:51.539663 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.539620    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4j287" podStartSLOduration=2.260771153 podStartE2EDuration="22.539603322s" totalImagesPullingTime="20.278832169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:51.539522747 +0000 UTC m=+22.756300208" watchObservedRunningTime="2026-07-29 21:16:51.539603322 +0000 UTC m=+22.756380784"
Jul 29 21:16:51.581756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.581709    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fvq2g" podStartSLOduration=2.197848246 podStartE2EDuration="22.581690781s" totalImagesPullingTime="20.383842535s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:51.581194749 +0000 UTC m=+22.797972212" watchObservedRunningTime="2026-07-29 21:16:51.581690781 +0000 UTC m=+22.798468243"
Jul 29 21:16:51.598711 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.598665    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qwjd4" podStartSLOduration=2.351534398 podStartE2EDuration="22.59864571s" totalImagesPullingTime="20.247111312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:51.598333684 +0000 UTC m=+22.815111146" watchObservedRunningTime="2026-07-29 21:16:51.59864571 +0000 UTC m=+22.815423172"
Jul 29 21:16:51.616821 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.616774    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-242.ec2.internal" podStartSLOduration=21.616759075 podStartE2EDuration="21.616759075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:51.616499143 +0000 UTC m=+22.833276605" watchObservedRunningTime="2026-07-29 21:16:51.616759075 +0000 UTC m=+22.833536536"
Jul 29 21:16:51.634236 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:51.634155    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5vkp2" podStartSLOduration=2.324023681 podStartE2EDuration="22.634134893s" totalImagesPullingTime="20.310111212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:51.634027966 +0000 UTC m=+22.850805428" watchObservedRunningTime="2026-07-29 21:16:51.634134893 +0000 UTC m=+22.850912356"
Jul 29 21:16:52.248981 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.248946    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:52.249625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.249605    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:52.298493 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.298459    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:52.298984 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.298459    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:52.298984 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:52.298601    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:52.298984 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:52.298678    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:52.515276 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.515233    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"96e5cc08be563ed6e1c708c8db3790e36c417d5320c0c2db6b5b2e3c9a4512ee"}
Jul 29 21:16:52.515452 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.515331    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"cfaa0d4e8f8ed60306a2f69aafbb13cde90458761636925033decee387643870"}
Jul 29 21:16:52.516820 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.516790    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8kpz" event={"ID":"d81690a6-3ab0-437d-83ce-1ac0e92c7db9","Type":"ContainerStarted","Data":"dfebdd750d70cdd769579532b0dfc139ea2f702fd03ffb66c6514101b44c33e7"}
Jul 29 21:16:52.518670 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.518641    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal" event={"ID":"d4b1019ff0d03c1e3fd7694711ab7fc6","Type":"ContainerStarted","Data":"d37c87fb502d6c06335d068bfbdbf32e20e56fefa05f3bd57cfe57f62f9b2757"}
Jul 29 21:16:52.519426 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.519404    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:52.520163 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.520072    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5vkp2"
Jul 29 21:16:52.535811 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.535764    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f8kpz" podStartSLOduration=3.254684325 podStartE2EDuration="23.535746207s" totalImagesPullingTime="20.281061882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:52.535468297 +0000 UTC m=+23.752245760" watchObservedRunningTime="2026-07-29 21:16:52.535746207 +0000 UTC m=+23.752523692"
Jul 29 21:16:52.575246 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.575189    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-242.ec2.internal" podStartSLOduration=22.575150849 podStartE2EDuration="22.575150849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:52.553482998 +0000 UTC m=+23.770260461" watchObservedRunningTime="2026-07-29 21:16:52.575150849 +0000 UTC m=+23.791928303"
Jul 29 21:16:52.648479 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:52.648455    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 21:16:53.197523 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:53.197417    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T21:16:52.64847473Z","UUID":"f0f61ac6-5605-4758-aa77-0f7ad5e66f5b","Handler":null,"Name":"","Endpoint":""}
Jul 29 21:16:53.199249 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:53.199225    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 21:16:53.199436 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:53.199272    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 21:16:53.297904 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:53.297868    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:53.298079 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:53.298004    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:53.524297 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:53.524207    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms" event={"ID":"749a806a-7d33-41cd-9f4d-3c262c8d7f80","Type":"ContainerStarted","Data":"1c2832d84f5540de3aa4538361bbfacea5c8b634ea3fa6e419726054cfe1f384"}
Jul 29 21:16:54.298070 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:54.298033    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:54.298274 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:54.298165    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:54.298274 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:54.298223    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:54.298403 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:54.298340    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:54.530758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:54.530726    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"e0f96b1940c242da5f3b3af44045adf82c09d28ee3e32905cf2cd7e8649722b4"}
Jul 29 21:16:55.298035 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:55.298002    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:55.298244 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:55.298140    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:56.298325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.298251    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:56.298325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.298292    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:56.299022 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:56.298358    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:56.299022 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:56.298470    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:56.538233 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.538190    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" event={"ID":"218e972a-89b8-4c61-82e0-d58efb52e1f9","Type":"ContainerStarted","Data":"fc9ceb85f1c3796695d83fc6aea79b0340196bc96d3bc1dd22eb352eaf49fa8b"}
Jul 29 21:16:56.538442 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.538425    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:56.540307 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.540283    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca902c68-ff65-4b9b-8b08-dcb3e36e1d97" containerID="2cd7a406c70a427f57f31cc6feaf2460f618d26f6cb51b96059999d851410029" exitCode=0
Jul 29 21:16:56.540423 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.540351    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerDied","Data":"2cd7a406c70a427f57f31cc6feaf2460f618d26f6cb51b96059999d851410029"}
Jul 29 21:16:56.542527 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.542507    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms" event={"ID":"749a806a-7d33-41cd-9f4d-3c262c8d7f80","Type":"ContainerStarted","Data":"279fc5edb32cdf14f373444f23b33faffd2c1952c3729186f4c7ce2b9f477011"}
Jul 29 21:16:56.554569 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.554519    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:56.567238 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.567199    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl" podStartSLOduration=7.311272762 podStartE2EDuration="27.567187419s" totalImagesPullingTime="20.255914657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:56.566700731 +0000 UTC m=+27.783478192" watchObservedRunningTime="2026-07-29 21:16:56.567187419 +0000 UTC m=+27.783964878"
Jul 29 21:16:56.616365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:56.616309    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-87xms" podStartSLOduration=2.928183487 podStartE2EDuration="27.616292553s" totalImagesPullingTime="24.688109066s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:16:56.616257309 +0000 UTC m=+27.833034771" watchObservedRunningTime="2026-07-29 21:16:56.616292553 +0000 UTC m=+27.833070015"
Jul 29 21:16:57.298767 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:57.298477    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:57.299591 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:57.298857    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:57.544769 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:57.544732    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:57.544968 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:57.544816    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:57.559609 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:57.559530    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:16:58.298460 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.298428    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:58.298460 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.298458    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:58.298707 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:58.298564    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:16:58.298707 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:58.298581    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:58.321239 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.321210    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqk4m"]
Jul 29 21:16:58.321683 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.321310    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:16:58.321683 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:58.321409    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:16:58.321865 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.321847    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8g669"]
Jul 29 21:16:58.323780 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.323755    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f7ghg"]
Jul 29 21:16:58.548453 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.548423    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca902c68-ff65-4b9b-8b08-dcb3e36e1d97" containerID="b4b4613557396d45ba5f4847f672b5c1a3b047a767a9ea96915baea17d56438e" exitCode=0
Jul 29 21:16:58.548625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.548528    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerDied","Data":"b4b4613557396d45ba5f4847f672b5c1a3b047a767a9ea96915baea17d56438e"}
Jul 29 21:16:58.548625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.548578    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:16:58.548753 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:16:58.548674    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:16:58.549020 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:58.548993    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:16:58.549134 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:16:58.549077    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:17:00.297665 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:00.297497    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:00.297950 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:00.297500    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:00.297950 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:00.297747    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:17:00.297950 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:00.297532    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:00.297950 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:00.297872    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:17:00.297950 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:00.297801    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:17:00.556018 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:00.555981    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca902c68-ff65-4b9b-8b08-dcb3e36e1d97" containerID="60e4a5c811196cdbf4f183afe0eccd02180ca0402144abefdf484b9a8b3f51fb" exitCode=0
Jul 29 21:17:00.556211 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:00.556036    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerDied","Data":"60e4a5c811196cdbf4f183afe0eccd02180ca0402144abefdf484b9a8b3f51fb"}
Jul 29 21:17:01.895907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:01.895864    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:01.896540 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.896037    2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 21:17:01.896540 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.896120    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret podName:3921626f-33f3-4c92-bdbe-c69a9ae6e54a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:33.896096664 +0000 UTC m=+65.112874121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret") pod "global-pull-secret-syncer-f7ghg" (UID: "3921626f-33f3-4c92-bdbe-c69a9ae6e54a") : object "kube-system"/"original-pull-secret" not registered
Jul 29 21:17:01.997134 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:01.997093    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:01.997307 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:01.997181    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:01.997307 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.997264    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:17:01.997307 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.997279    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 21:17:01.997307 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.997295    2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 21:17:01.997307 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.997307    2572 projected.go:196] Error preparing data for projected volume kube-api-access-4nzhr for pod openshift-network-diagnostics/network-check-target-jqk4m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:17:01.997522 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.997338    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs podName:46824038-d39b-48cd-9bd4-32177f2500f2 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:33.997317421 +0000 UTC m=+65.214094875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs") pod "network-metrics-daemon-8g669" (UID: "46824038-d39b-48cd-9bd4-32177f2500f2") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 21:17:01.997522 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:01.997354    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr podName:21463221-4b14-49d1-99d0-686a9b61c757 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:33.99734688 +0000 UTC m=+65.214124319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4nzhr" (UniqueName: "kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr") pod "network-check-target-jqk4m" (UID: "21463221-4b14-49d1-99d0-686a9b61c757") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 21:17:02.298101 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:02.298072    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:02.298284 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:02.298072    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:02.298284 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:02.298073    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:02.298284 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:02.298270    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:17:02.298445 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:02.298357    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:17:02.298445 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:02.298417    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:17:02.342235 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:02.342198    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:02.445807 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:02.445774    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:03.417672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:03.417637    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:04.298006 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.297973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:04.298205 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.297973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:04.298205 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:04.298100    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f7ghg" podUID="3921626f-33f3-4c92-bdbe-c69a9ae6e54a"
Jul 29 21:17:04.298205 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.297973    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:04.298205 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:04.298163    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8g669" podUID="46824038-d39b-48cd-9bd4-32177f2500f2"
Jul 29 21:17:04.298396 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:04.298286    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqk4m" podUID="21463221-4b14-49d1-99d0-686a9b61c757"
Jul 29 21:17:04.499191 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.499140    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:04.591882 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.591849    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:04.617850 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.617814    2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-242.ec2.internal" event="NodeReady"
Jul 29 21:17:04.618016 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.618001    2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 21:17:04.668626 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.668590    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m6l9g"]
Jul 29 21:17:04.670748 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.670724    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-p6jlr"]
Jul 29 21:17:04.670867 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.670849    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.672739 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.672720    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:04.673626 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.673605    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 21:17:04.673735 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.673642    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 21:17:04.673735 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.673729    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6msdl\""
Jul 29 21:17:04.675992 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.675971    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 21:17:04.676098 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.676044    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-djnws\""
Jul 29 21:17:04.676098 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.675971    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 21:17:04.676244 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.676094    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 21:17:04.682361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.682338    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p6jlr"]
Jul 29 21:17:04.685625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.685606    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m6l9g"]
Jul 29 21:17:04.819488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.819389    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/956e2318-8d5c-46db-9e95-5f0ff311393b-config-volume\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.819488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.819451    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8sql\" (UniqueName: \"kubernetes.io/projected/ff96152c-cdba-4425-98b2-3b13d935763e-kube-api-access-t8sql\") pod \"ingress-canary-p6jlr\" (UID: \"ff96152c-cdba-4425-98b2-3b13d935763e\") " pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:04.819720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.819524    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff96152c-cdba-4425-98b2-3b13d935763e-cert\") pod \"ingress-canary-p6jlr\" (UID: \"ff96152c-cdba-4425-98b2-3b13d935763e\") " pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:04.819720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.819570    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/956e2318-8d5c-46db-9e95-5f0ff311393b-tmp-dir\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.819720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.819634    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.819720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.819698    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrp9d\" (UniqueName: \"kubernetes.io/projected/956e2318-8d5c-46db-9e95-5f0ff311393b-kube-api-access-lrp9d\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.920278 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920250    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/956e2318-8d5c-46db-9e95-5f0ff311393b-config-volume\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.920413 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8sql\" (UniqueName: \"kubernetes.io/projected/ff96152c-cdba-4425-98b2-3b13d935763e-kube-api-access-t8sql\") pod \"ingress-canary-p6jlr\" (UID: \"ff96152c-cdba-4425-98b2-3b13d935763e\") " pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:04.920413 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920308    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff96152c-cdba-4425-98b2-3b13d935763e-cert\") pod \"ingress-canary-p6jlr\" (UID: \"ff96152c-cdba-4425-98b2-3b13d935763e\") " pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:04.920413 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920330    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/956e2318-8d5c-46db-9e95-5f0ff311393b-tmp-dir\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.920413 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.920576 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920407    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrp9d\" (UniqueName: \"kubernetes.io/projected/956e2318-8d5c-46db-9e95-5f0ff311393b-kube-api-access-lrp9d\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.920576 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:04.920526    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:17:04.920643 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:04.920615    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls podName:956e2318-8d5c-46db-9e95-5f0ff311393b nodeName:}" failed. No retries permitted until 2026-07-29 21:17:05.420590882 +0000 UTC m=+36.637368322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls") pod "dns-default-m6l9g" (UID: "956e2318-8d5c-46db-9e95-5f0ff311393b") : secret "dns-default-metrics-tls" not found
Jul 29 21:17:04.920720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920701    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/956e2318-8d5c-46db-9e95-5f0ff311393b-tmp-dir\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.920900 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.920880    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/956e2318-8d5c-46db-9e95-5f0ff311393b-config-volume\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.924645 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.924623    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff96152c-cdba-4425-98b2-3b13d935763e-cert\") pod \"ingress-canary-p6jlr\" (UID: \"ff96152c-cdba-4425-98b2-3b13d935763e\") " pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:04.929562 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.929536    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrp9d\" (UniqueName: \"kubernetes.io/projected/956e2318-8d5c-46db-9e95-5f0ff311393b-kube-api-access-lrp9d\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:04.929950 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:04.929933    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8sql\" (UniqueName: \"kubernetes.io/projected/ff96152c-cdba-4425-98b2-3b13d935763e-kube-api-access-t8sql\") pod \"ingress-canary-p6jlr\" (UID: \"ff96152c-cdba-4425-98b2-3b13d935763e\") " pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:05.015071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.015022    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-p6jlr"
Jul 29 21:17:05.144500 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.144470    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-p6jlr"]
Jul 29 21:17:05.147909 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:05.147883    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff96152c_cdba_4425_98b2_3b13d935763e.slice/crio-2e218b3670c465c055622a2309fb1164994589055e60f985ae2ff5e99c99b030 WatchSource:0}: Error finding container 2e218b3670c465c055622a2309fb1164994589055e60f985ae2ff5e99c99b030: Status 404 returned error can't find the container with id 2e218b3670c465c055622a2309fb1164994589055e60f985ae2ff5e99c99b030
Jul 29 21:17:05.423923 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.423835    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:05.424073 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:05.423956    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:17:05.424073 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:05.424019    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls podName:956e2318-8d5c-46db-9e95-5f0ff311393b nodeName:}" failed. No retries permitted until 2026-07-29 21:17:06.424005273 +0000 UTC m=+37.640782716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls") pod "dns-default-m6l9g" (UID: "956e2318-8d5c-46db-9e95-5f0ff311393b") : secret "dns-default-metrics-tls" not found
Jul 29 21:17:05.574295 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.574262    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca902c68-ff65-4b9b-8b08-dcb3e36e1d97" containerID="d5d714d0342fecc97f15900896c5bba66e559f35ac1ede375d73864371fa1b2a" exitCode=0
Jul 29 21:17:05.574853 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.574345    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerDied","Data":"d5d714d0342fecc97f15900896c5bba66e559f35ac1ede375d73864371fa1b2a"}
Jul 29 21:17:05.575708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.575446    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p6jlr" event={"ID":"ff96152c-cdba-4425-98b2-3b13d935763e","Type":"ContainerStarted","Data":"2e218b3670c465c055622a2309fb1164994589055e60f985ae2ff5e99c99b030"}
Jul 29 21:17:05.607938 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.607901    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:05.705305 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:05.705275    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:06.297984 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.297946    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:06.298160 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.297985    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:06.298160 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.297985    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:06.300403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.300380    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 21:17:06.300505 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.300428    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 21:17:06.301108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.301057    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ql2h5\""
Jul 29 21:17:06.301108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.301084    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 21:17:06.301335 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.301127    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-929b2\""
Jul 29 21:17:06.301335 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.301127    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 21:17:06.433123 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.433088    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:06.433290 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:06.433260    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:17:06.433343 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:06.433332    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls podName:956e2318-8d5c-46db-9e95-5f0ff311393b nodeName:}" failed. No retries permitted until 2026-07-29 21:17:08.43331077 +0000 UTC m=+39.650088216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls") pod "dns-default-m6l9g" (UID: "956e2318-8d5c-46db-9e95-5f0ff311393b") : secret "dns-default-metrics-tls" not found
Jul 29 21:17:06.582153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.582064    2572 generic.go:356] "Generic (PLEG): container finished" podID="ca902c68-ff65-4b9b-8b08-dcb3e36e1d97" containerID="c5e7fb7af9839b343fc324ecbe81c0444d7a9a7e92621abec785cb429892ccf4" exitCode=0
Jul 29 21:17:06.582153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.582144    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerDied","Data":"c5e7fb7af9839b343fc324ecbe81c0444d7a9a7e92621abec785cb429892ccf4"}
Jul 29 21:17:06.890201 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.890099    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zd96v"]
Jul 29 21:17:06.894693 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.894663    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:06.899107 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.899087    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mr6st\""
Jul 29 21:17:06.899570 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.899548    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 21:17:06.899678 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.899608    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 21:17:06.899745 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.899679    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 21:17:06.899745 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.899680    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 21:17:06.904005 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.903986    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 21:17:06.912538 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.912513    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zd96v"]
Jul 29 21:17:06.981714 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.981681    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"]
Jul 29 21:17:06.999790 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.999750    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"]
Jul 29 21:17:06.999959 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:06.999876    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.002386 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.002361    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 21:17:07.002572 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.002367    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:17:07.003185 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.003020    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 21:17:07.003185 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.003164    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 21:17:07.003358 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.003251    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6x2r7\""
Jul 29 21:17:07.012250 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.012228    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"]
Jul 29 21:17:07.012380 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.012333    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.014606 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.014563    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 21:17:07.014727 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.014571    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:17:07.014727 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.014647    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 21:17:07.014727 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.014685    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pqgnt\""
Jul 29 21:17:07.014930 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.014914    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 21:17:07.034671 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.034639    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"]
Jul 29 21:17:07.034824 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.034682    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"]
Jul 29 21:17:07.034824 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.034704    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"]
Jul 29 21:17:07.034824 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.034784    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.037348 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.037323    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 21:17:07.037477 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.037335    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 21:17:07.037600 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.037579    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 21:17:07.037742 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.037594    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g5w5t\""
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.037986    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-service-ca-bundle\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.038068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-tmp\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.038102    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.038125    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-serving-cert\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.038218    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts5mq\" (UniqueName: \"kubernetes.io/projected/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-kube-api-access-ts5mq\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.038246    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 21:17:07.038333 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.038249    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-snapshots\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.139307 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139266    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-service-ca-bundle\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.139473 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139375    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-tmp\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.139473 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139405    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.139473 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139443    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.139473 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139465    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-serving-cert\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.139684 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139635    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.139741 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139679    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1deada3b-1099-4beb-b51f-b4839686a1a8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.139741 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139715    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-config\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.139839 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139745    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kprfm\" (UniqueName: \"kubernetes.io/projected/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-kube-api-access-kprfm\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.139839 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139766    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-tmp\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.139839 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139771    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1deada3b-1099-4beb-b51f-b4839686a1a8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.139839 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb5qz\" (UniqueName: \"kubernetes.io/projected/1deada3b-1099-4beb-b51f-b4839686a1a8-kube-api-access-tb5qz\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.140031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139859    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45e75ef5-8b16-4b9e-9715-d50c47da09f0-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.140031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139904    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts5mq\" (UniqueName: \"kubernetes.io/projected/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-kube-api-access-ts5mq\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.140031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.139935    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-service-ca-bundle\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.140031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.140021    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-snapshots\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.140270 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.140052    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl4d7\" (UniqueName: \"kubernetes.io/projected/45e75ef5-8b16-4b9e-9715-d50c47da09f0-kube-api-access-bl4d7\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.140270 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.140252    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.140611 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.140589    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-snapshots\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.144371 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.144346    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-serving-cert\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.148382 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.148257    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts5mq\" (UniqueName: \"kubernetes.io/projected/ddf60a13-3a1c-41fa-8212-6762ab9a37c6-kube-api-access-ts5mq\") pod \"insights-operator-5885fdb759-zd96v\" (UID: \"ddf60a13-3a1c-41fa-8212-6762ab9a37c6\") " pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.206401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.206353    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-zd96v"
Jul 29 21:17:07.232497 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.232313    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:07.241047 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241024    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45e75ef5-8b16-4b9e-9715-d50c47da09f0-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.241145 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241068    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl4d7\" (UniqueName: \"kubernetes.io/projected/45e75ef5-8b16-4b9e-9715-d50c47da09f0-kube-api-access-bl4d7\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.241205 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241160    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.241269 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241218    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.241269 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241236    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1deada3b-1099-4beb-b51f-b4839686a1a8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.241269 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241257    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-config\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.241410 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:07.241284    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:07.241410 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:07.241367    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls podName:45e75ef5-8b16-4b9e-9715-d50c47da09f0 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:07.741346608 +0000 UTC m=+38.958124066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-8vhcv" (UID: "45e75ef5-8b16-4b9e-9715-d50c47da09f0") : secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:07.241410 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241358    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kprfm\" (UniqueName: \"kubernetes.io/projected/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-kube-api-access-kprfm\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.241562 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241406    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1deada3b-1099-4beb-b51f-b4839686a1a8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.241562 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241437    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb5qz\" (UniqueName: \"kubernetes.io/projected/1deada3b-1099-4beb-b51f-b4839686a1a8-kube-api-access-tb5qz\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.241855 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241836    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-config\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.242001 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.241978    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/45e75ef5-8b16-4b9e-9715-d50c47da09f0-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.242184 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.242146    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1deada3b-1099-4beb-b51f-b4839686a1a8-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.243577 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.243558    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-serving-cert\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.243719 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.243698    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1deada3b-1099-4beb-b51f-b4839686a1a8-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.252964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.252931    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb5qz\" (UniqueName: \"kubernetes.io/projected/1deada3b-1099-4beb-b51f-b4839686a1a8-kube-api-access-tb5qz\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl\" (UID: \"1deada3b-1099-4beb-b51f-b4839686a1a8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.253208 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.253191    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kprfm\" (UniqueName: \"kubernetes.io/projected/b9ebc0f6-62d4-4f59-aea4-674376a9dab8-kube-api-access-kprfm\") pod \"service-ca-operator-5d6c75f84f-6x542\" (UID: \"b9ebc0f6-62d4-4f59-aea4-674376a9dab8\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.253711 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.253690    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl4d7\" (UniqueName: \"kubernetes.io/projected/45e75ef5-8b16-4b9e-9715-d50c47da09f0-kube-api-access-bl4d7\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.311444 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.311346    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"
Jul 29 21:17:07.322452 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.322428    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"
Jul 29 21:17:07.335157 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.335126    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:07.387465 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.387442    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-zd96v"]
Jul 29 21:17:07.429051 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:07.429013    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf60a13_3a1c_41fa_8212_6762ab9a37c6.slice/crio-eb99a4d95a0f175a183fa281d1525bd762df24a484ee39d73ce4d9540fd2e789 WatchSource:0}: Error finding container eb99a4d95a0f175a183fa281d1525bd762df24a484ee39d73ce4d9540fd2e789: Status 404 returned error can't find the container with id eb99a4d95a0f175a183fa281d1525bd762df24a484ee39d73ce4d9540fd2e789
Jul 29 21:17:07.572285 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.571965    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl"]
Jul 29 21:17:07.575421 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.575392    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542"]
Jul 29 21:17:07.575567 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:07.575427    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1deada3b_1099_4beb_b51f_b4839686a1a8.slice/crio-6b61e00208f2dfabc4270f46d3bf48ce83028d1d1eff8804ad92b7e73afb9121 WatchSource:0}: Error finding container 6b61e00208f2dfabc4270f46d3bf48ce83028d1d1eff8804ad92b7e73afb9121: Status 404 returned error can't find the container with id 6b61e00208f2dfabc4270f46d3bf48ce83028d1d1eff8804ad92b7e73afb9121
Jul 29 21:17:07.578462 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:07.578442    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ebc0f6_62d4_4f59_aea4_674376a9dab8.slice/crio-bc6d7aaf14772f5c34ddd85b972d6a88eabfffcc7f7095ae0ea77a3126e0bce3 WatchSource:0}: Error finding container bc6d7aaf14772f5c34ddd85b972d6a88eabfffcc7f7095ae0ea77a3126e0bce3: Status 404 returned error can't find the container with id bc6d7aaf14772f5c34ddd85b972d6a88eabfffcc7f7095ae0ea77a3126e0bce3
Jul 29 21:17:07.588929 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.588901    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zd96v" event={"ID":"ddf60a13-3a1c-41fa-8212-6762ab9a37c6","Type":"ContainerStarted","Data":"eb99a4d95a0f175a183fa281d1525bd762df24a484ee39d73ce4d9540fd2e789"}
Jul 29 21:17:07.590733 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.590703    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-p6jlr" event={"ID":"ff96152c-cdba-4425-98b2-3b13d935763e","Type":"ContainerStarted","Data":"24e76c34fe54b97c5417d9ba6017495149e3223d7ebb4e6d0805cf10faf6286e"}
Jul 29 21:17:07.594488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.594465    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" event={"ID":"ca902c68-ff65-4b9b-8b08-dcb3e36e1d97","Type":"ContainerStarted","Data":"d84911894cefbb4a34ff6c61eda82b1f30f74f97ca272ed7faa923a06b5bbc3d"}
Jul 29 21:17:07.595679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.595659    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542" event={"ID":"b9ebc0f6-62d4-4f59-aea4-674376a9dab8","Type":"ContainerStarted","Data":"bc6d7aaf14772f5c34ddd85b972d6a88eabfffcc7f7095ae0ea77a3126e0bce3"}
Jul 29 21:17:07.598401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.598378    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl" event={"ID":"1deada3b-1099-4beb-b51f-b4839686a1a8","Type":"ContainerStarted","Data":"6b61e00208f2dfabc4270f46d3bf48ce83028d1d1eff8804ad92b7e73afb9121"}
Jul 29 21:17:07.610653 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.610608    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-p6jlr" podStartSLOduration=1.318152113 podStartE2EDuration="3.610592054s" totalImagesPullingTime="2.292439941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:07.609724309 +0000 UTC m=+38.826501771" watchObservedRunningTime="2026-07-29 21:17:07.610592054 +0000 UTC m=+38.827369493"
Jul 29 21:17:07.635449 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.635401    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5zgsh" podStartSLOduration=7.201923711 podStartE2EDuration="38.63538549s" totalImagesPullingTime="31.433461779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:07.634588623 +0000 UTC m=+38.851366086" watchObservedRunningTime="2026-07-29 21:17:07.63538549 +0000 UTC m=+38.852162952"
Jul 29 21:17:07.745455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.745423    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:07.745642 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:07.745597    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:07.745706 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:07.745682    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls podName:45e75ef5-8b16-4b9e-9715-d50c47da09f0 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:08.745661474 +0000 UTC m=+39.962438935 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-8vhcv" (UID: "45e75ef5-8b16-4b9e-9715-d50c47da09f0") : secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:07.760320 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.760290    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:07.860334 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:07.860304    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:08.451223 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:08.451126    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:08.451415 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:08.451285    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:17:08.451415 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:08.451348    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls podName:956e2318-8d5c-46db-9e95-5f0ff311393b nodeName:}" failed. No retries permitted until 2026-07-29 21:17:12.451329917 +0000 UTC m=+43.668107358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls") pod "dns-default-m6l9g" (UID: "956e2318-8d5c-46db-9e95-5f0ff311393b") : secret "dns-default-metrics-tls" not found
Jul 29 21:17:08.753981 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:08.753895    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:08.754570 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:08.754548    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:08.754651 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:08.754626    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls podName:45e75ef5-8b16-4b9e-9715-d50c47da09f0 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:10.754602125 +0000 UTC m=+41.971379582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-8vhcv" (UID: "45e75ef5-8b16-4b9e-9715-d50c47da09f0") : secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:08.988489 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:08.988438    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:09.090762 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:09.090654    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:09.598926 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:09.598885    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:09.672978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:09.672944    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:10.489915 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:10.489877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:10.579152 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:10.579121    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:10.770968 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:10.770886    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:10.771115 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:10.771048    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:10.771115 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:10.771112    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls podName:45e75ef5-8b16-4b9e-9715-d50c47da09f0 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:14.77109637 +0000 UTC m=+45.987873810 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-8vhcv" (UID: "45e75ef5-8b16-4b9e-9715-d50c47da09f0") : secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:12.484523 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.484470    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:12.484925 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:12.484620    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:17:12.484925 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:12.484688    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls podName:956e2318-8d5c-46db-9e95-5f0ff311393b nodeName:}" failed. No retries permitted until 2026-07-29 21:17:20.484669027 +0000 UTC m=+51.701446470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls") pod "dns-default-m6l9g" (UID: "956e2318-8d5c-46db-9e95-5f0ff311393b") : secret "dns-default-metrics-tls" not found
Jul 29 21:17:12.613616 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.613573    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl" event={"ID":"1deada3b-1099-4beb-b51f-b4839686a1a8","Type":"ContainerStarted","Data":"9199059e363646cd870b24c3f2924bd83f57b233db24e3a01928aca14a5adbf5"}
Jul 29 21:17:12.615234 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.615202    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zd96v" event={"ID":"ddf60a13-3a1c-41fa-8212-6762ab9a37c6","Type":"ContainerStarted","Data":"042d4b5d8147811b0060d990574e5eae21cddbbee349b1956a54a03290a1d6a4"}
Jul 29 21:17:12.616786 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.616762    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542" event={"ID":"b9ebc0f6-62d4-4f59-aea4-674376a9dab8","Type":"ContainerStarted","Data":"cc502841e8b8761f5420a373ca1fa3fc7b3289bcf0366646d8d530e59fa4cba4"}
Jul 29 21:17:12.631125 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.631078    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl" podStartSLOduration=2.427040938 podStartE2EDuration="6.631064826s" totalImagesPullingTime="4.204023888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:12.630032455 +0000 UTC m=+43.846809918" watchObservedRunningTime="2026-07-29 21:17:12.631064826 +0000 UTC m=+43.847842303"
Jul 29 21:17:12.647782 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.647716    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542" podStartSLOduration=2.445798474 podStartE2EDuration="6.647694541s" totalImagesPullingTime="4.201896067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:12.646973484 +0000 UTC m=+43.863750946" watchObservedRunningTime="2026-07-29 21:17:12.647694541 +0000 UTC m=+43.864471981"
Jul 29 21:17:12.669883 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.669782    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-zd96v" podStartSLOduration=2.316424952 podStartE2EDuration="6.669758693s" totalImagesPullingTime="4.353333741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:12.66938368 +0000 UTC m=+43.886161158" watchObservedRunningTime="2026-07-29 21:17:12.669758693 +0000 UTC m=+43.886536158"
Jul 29 21:17:12.814613 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.814541    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mmmvr"]
Jul 29 21:17:12.839470 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.839434    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:12.845633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.845600    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mmmvr"]
Jul 29 21:17:12.845833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.845731    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:12.848014 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.847988    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 21:17:12.848158 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.848095    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 21:17:12.848158 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.848118    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m4rkn\""
Jul 29 21:17:12.912338 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.912302    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:12.989402 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.989357    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:12.989566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.989415    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-crio-socket\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:12.989566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.989445    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-data-volume\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:12.989566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.989518    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwkmj\" (UniqueName: \"kubernetes.io/projected/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-api-access-lwkmj\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:12.989721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:12.989615    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091079 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.090988    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwkmj\" (UniqueName: \"kubernetes.io/projected/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-api-access-lwkmj\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091257 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.091092    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091257 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.091145    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091257 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.091180    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:17:13.091442 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.091267    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:13.591247056 +0000 UTC m=+44.808024510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : configmap "kube-rbac-proxy" not found
Jul 29 21:17:13.091442 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.091192    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-crio-socket\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091442 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.091302    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:13.091442 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.091319    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-data-volume\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091442 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.091368    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:13.591352084 +0000 UTC m=+44.808129536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:13.091442 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.091364    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-crio-socket\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.091743 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.091658    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-data-volume\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.101262 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.101235    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwkmj\" (UniqueName: \"kubernetes.io/projected/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-api-access-lwkmj\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.232259 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.232220    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:13.288739 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.288699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:13.517242 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.517196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:13.597300 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.597267    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.597471 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.597429    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:13.597516 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.597485    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:13.597516 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.597506    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:14.597486786 +0000 UTC m=+45.814264249 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:13.597593 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.597542    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:17:13.597593 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:13.597579    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:14.597568633 +0000 UTC m=+45.814346078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : configmap "kube-rbac-proxy" not found
Jul 29 21:17:13.606505 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.606477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:13.708389 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.708357    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:13.773810 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.773731    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:13.944311 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:13.944258    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:14.002256 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:14.002221    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:14.595765 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:14.595740    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4j287_d0679383-8d53-4278-958c-b66a18435ce0/dns-node-resolver/0.log"
Jul 29 21:17:14.605104 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:14.605078    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:14.605260 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:14.605120    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:14.605260 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:14.605222    2572 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 21:17:14.605260 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:14.605245    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:14.605372 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:14.605279    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:16.60526382 +0000 UTC m=+47.822041261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : configmap "kube-rbac-proxy" not found
Jul 29 21:17:14.605372 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:14.605295    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:16.605289197 +0000 UTC m=+47.822066638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:14.806908 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:14.806874    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:14.807071 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:14.806993    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:14.807071 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:14.807046    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls podName:45e75ef5-8b16-4b9e-9715-d50c47da09f0 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:22.807033028 +0000 UTC m=+54.023810469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-8vhcv" (UID: "45e75ef5-8b16-4b9e-9715-d50c47da09f0") : secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:14.900675 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:14.900596    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:15.564121 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.564085    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-qhswz"]
Jul 29 21:17:15.594816 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.594792    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qsc5x_c54c0b98-fe67-4c62-9663-7a21dccf869b/node-ca/0.log"
Jul 29 21:17:15.605125 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.605093    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-qhswz"]
Jul 29 21:17:15.605456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.605228    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.606805 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.606765    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:15.607626 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.607605    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 29 21:17:15.607734 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.607663    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 29 21:17:15.607734 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.607724    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-phszt\""
Jul 29 21:17:15.608438 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.608422    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 29 21:17:15.608595 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.608584    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 29 21:17:15.683732 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.683704    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"]
Jul 29 21:17:15.685466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.685442    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:15.704582 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.704558    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:15.708623 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.708603    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-g4rzc\""
Jul 29 21:17:15.708746 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.708638    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 29 21:17:15.708746 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.708716    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 29 21:17:15.712050 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.712026    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"]
Jul 29 21:17:15.713104 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.713084    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5blcr\" (UniqueName: \"kubernetes.io/projected/854c378c-0684-4a24-8961-181c2c9ef206-kube-api-access-5blcr\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.713227 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.713145    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/854c378c-0684-4a24-8961-181c2c9ef206-signing-cabundle\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.713227 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.713163    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/854c378c-0684-4a24-8961-181c2c9ef206-signing-key\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.814109 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.814070    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:15.814342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.814209    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:15.814342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.814270    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5blcr\" (UniqueName: \"kubernetes.io/projected/854c378c-0684-4a24-8961-181c2c9ef206-kube-api-access-5blcr\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.814342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.814338    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/854c378c-0684-4a24-8961-181c2c9ef206-signing-cabundle\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.814482 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.814358    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/854c378c-0684-4a24-8961-181c2c9ef206-signing-key\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.815071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.815047    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/854c378c-0684-4a24-8961-181c2c9ef206-signing-cabundle\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.816622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.816600    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/854c378c-0684-4a24-8961-181c2c9ef206-signing-key\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.822435 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.822413    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5blcr\" (UniqueName: \"kubernetes.io/projected/854c378c-0684-4a24-8961-181c2c9ef206-kube-api-access-5blcr\") pod \"service-ca-658d7b7c78-qhswz\" (UID: \"854c378c-0684-4a24-8961-181c2c9ef206\") " pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.915039 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.915001    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-qhswz"
Jul 29 21:17:15.915240 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.915221    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:15.915289 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.915279    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:15.915391 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:15.915381    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 21:17:15.915439 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:15.915431    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert podName:416db6ee-0be2-4e75-95c7-bd3d2da60d8a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:16.415416717 +0000 UTC m=+47.632194157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-q4csj" (UID: "416db6ee-0be2-4e75-95c7-bd3d2da60d8a") : secret "networking-console-plugin-cert" not found
Jul 29 21:17:15.915803 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.915784    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:15.969336 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:15.969309    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:16.038146 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.038104    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:16.070612 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.070540    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-qhswz"]
Jul 29 21:17:16.074721 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:16.074694    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod854c378c_0684_4a24_8961_181c2c9ef206.slice/crio-acab221ed65cffaeb0775259537fe1957a3bfad54260952f75bf33769520ad70 WatchSource:0}: Error finding container acab221ed65cffaeb0775259537fe1957a3bfad54260952f75bf33769520ad70: Status 404 returned error can't find the container with id acab221ed65cffaeb0775259537fe1957a3bfad54260952f75bf33769520ad70
Jul 29 21:17:16.395686 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.395600    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p6jlr_ff96152c-cdba-4425-98b2-3b13d935763e/serve-healthcheck-canary/0.log"
Jul 29 21:17:16.420017 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.419970    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:16.420195 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:16.420116    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 21:17:16.420195 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:16.420193    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert podName:416db6ee-0be2-4e75-95c7-bd3d2da60d8a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:17.420176634 +0000 UTC m=+48.636954088 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-q4csj" (UID: "416db6ee-0be2-4e75-95c7-bd3d2da60d8a") : secret "networking-console-plugin-cert" not found
Jul 29 21:17:16.621913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.621872    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:16.622415 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:16.622054    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:16.622415 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.622071    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:16.622415 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:16.622132    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:20.622112054 +0000 UTC m=+51.838889514 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:16.622702 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.622532    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:16.631923 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.631897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-qhswz" event={"ID":"854c378c-0684-4a24-8961-181c2c9ef206","Type":"ContainerStarted","Data":"ee1fc820b1e8739563bbd0ed5dd2f6392425ed1c9938e0ab6313f60969400737"}
Jul 29 21:17:16.632056 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.631961    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-qhswz" event={"ID":"854c378c-0684-4a24-8961-181c2c9ef206","Type":"ContainerStarted","Data":"acab221ed65cffaeb0775259537fe1957a3bfad54260952f75bf33769520ad70"}
Jul 29 21:17:16.698897 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:16.698846    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:17.428624 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:17.428576    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:17.428852 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:17.428837    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 21:17:17.428924 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:17.428905    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert podName:416db6ee-0be2-4e75-95c7-bd3d2da60d8a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:19.428885827 +0000 UTC m=+50.645663273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-q4csj" (UID: "416db6ee-0be2-4e75-95c7-bd3d2da60d8a") : secret "networking-console-plugin-cert" not found
Jul 29 21:17:17.491219 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:17.491184    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:17.591770 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:17.591725    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:17.695380 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:17.695280    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:17.793300 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:17.793264    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:17.893678 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:17.893640    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:18.563226 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:18.563192    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:18.968472 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:18.968434    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:19.374081 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:19.373983    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:19.444230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:19.444192    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:19.444380 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:19.444347    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 21:17:19.444432 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:19.444422    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert podName:416db6ee-0be2-4e75-95c7-bd3d2da60d8a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:23.444404989 +0000 UTC m=+54.661182429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-q4csj" (UID: "416db6ee-0be2-4e75-95c7-bd3d2da60d8a") : secret "networking-console-plugin-cert" not found
Jul 29 21:17:19.578866 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:19.578817    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:20.163988 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.163940    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:20.227856 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.227817    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:20.497901 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.497860    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:20.551916 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.551889    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:20.552111 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:20.552001    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 21:17:20.552111 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:20.552078    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls podName:956e2318-8d5c-46db-9e95-5f0ff311393b nodeName:}" failed. No retries permitted until 2026-07-29 21:17:36.552060699 +0000 UTC m=+67.768838139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls") pod "dns-default-m6l9g" (UID: "956e2318-8d5c-46db-9e95-5f0ff311393b") : secret "dns-default-metrics-tls" not found
Jul 29 21:17:20.597949 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.597906    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:20.652327 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.652296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:20.652521 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:20.652459    2572 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:20.652592 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:20.652535    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls podName:b6dc41d9-5eaa-482c-822a-b9e18c2f6b04 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:28.65251464 +0000 UTC m=+59.869292093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mmmvr" (UID: "b6dc41d9-5eaa-482c-822a-b9e18c2f6b04") : secret "insights-runtime-extractor-tls" not found
Jul 29 21:17:20.764061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:20.763954    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:21.170683 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:21.170581    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:21.235281 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:21.235245    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:21.654940 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:21.654904    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:21.769479 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:21.769435    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:22.262266 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:22.262228    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:22.379902 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:22.379871    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:22.688031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:22.688000    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:22.790382 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:22.790354    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:22.869111 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:22.869062    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:22.869335 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:22.869250    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:22.869335 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:22.869332    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls podName:45e75ef5-8b16-4b9e-9715-d50c47da09f0 nodeName:}" failed. No retries permitted until 2026-07-29 21:17:38.869311154 +0000 UTC m=+70.086088599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-8vhcv" (UID: "45e75ef5-8b16-4b9e-9715-d50c47da09f0") : secret "cluster-monitoring-operator-tls" not found
Jul 29 21:17:23.365919 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:23.365880    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:23.473980 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:23.473933    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:23.474150 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:23.474098    2572 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 29 21:17:23.474222 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:23.474164    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert podName:416db6ee-0be2-4e75-95c7-bd3d2da60d8a nodeName:}" failed. No retries permitted until 2026-07-29 21:17:31.474145133 +0000 UTC m=+62.690922582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-q4csj" (UID: "416db6ee-0be2-4e75-95c7-bd3d2da60d8a") : secret "networking-console-plugin-cert" not found
Jul 29 21:17:23.773711 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:23.773673    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:24.170076 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:24.169983    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:24.573555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:24.573519    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:24.741121 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:24.741086    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:24.812759 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:24.812725    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:25.169875 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:25.169831    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:25.982506 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:25.982477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:26.364834 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:26.364761    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:26.568947 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:26.568919    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:26.764869 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:26.764838    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:26.970155 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:26.970117    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:27.765625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:27.765581    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:27.970028 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:27.969991    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:28.161948 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.161851    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:28.228251 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.228220    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:28.717406 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.717375    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:28.719677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.719658    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b6dc41d9-5eaa-482c-822a-b9e18c2f6b04-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mmmvr\" (UID: \"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04\") " pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:28.756866 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.756837    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mmmvr"
Jul 29 21:17:28.893190 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.893097    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-qhswz" podStartSLOduration=13.893075979 podStartE2EDuration="13.893075979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:16.661709455 +0000 UTC m=+47.878486927" watchObservedRunningTime="2026-07-29 21:17:28.893075979 +0000 UTC m=+60.109853442"
Jul 29 21:17:28.893661 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:28.893643    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mmmvr"]
Jul 29 21:17:28.897043 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:28.897019    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6dc41d9_5eaa_482c_822a_b9e18c2f6b04.slice/crio-c10b3068f1390b0136401e7e86a317f5ab9cc5eb733e92eaf1e671e1e5fbd5ac WatchSource:0}: Error finding container c10b3068f1390b0136401e7e86a317f5ab9cc5eb733e92eaf1e671e1e5fbd5ac: Status 404 returned error can't find the container with id c10b3068f1390b0136401e7e86a317f5ab9cc5eb733e92eaf1e671e1e5fbd5ac
Jul 29 21:17:29.567684 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:29.567655    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9nwdl"
Jul 29 21:17:29.667456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:29.667331    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmmvr" event={"ID":"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04","Type":"ContainerStarted","Data":"c10b3068f1390b0136401e7e86a317f5ab9cc5eb733e92eaf1e671e1e5fbd5ac"}
Jul 29 21:17:31.538189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:31.538135    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:31.540545 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:31.540514    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/416db6ee-0be2-4e75-95c7-bd3d2da60d8a-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-q4csj\" (UID: \"416db6ee-0be2-4e75-95c7-bd3d2da60d8a\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:31.618040 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:31.618009    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-g4rzc\""
Jul 29 21:17:31.624135 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:31.624106    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"
Jul 29 21:17:33.958695 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:33.958650    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:33.961541 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:33.961503    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 21:17:33.971254 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:33.971228    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3921626f-33f3-4c92-bdbe-c69a9ae6e54a-original-pull-secret\") pod \"global-pull-secret-syncer-f7ghg\" (UID: \"3921626f-33f3-4c92-bdbe-c69a9ae6e54a\") " pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:34.059808 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.059768    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:34.060002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.059819    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:34.062536 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.062376    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 21:17:34.062943 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.062913    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 21:17:34.073003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.072972    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 21:17:34.073345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.073323    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46824038-d39b-48cd-9bd4-32177f2500f2-metrics-certs\") pod \"network-metrics-daemon-8g669\" (UID: \"46824038-d39b-48cd-9bd4-32177f2500f2\") " pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:34.083645 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.083617    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nzhr\" (UniqueName: \"kubernetes.io/projected/21463221-4b14-49d1-99d0-686a9b61c757-kube-api-access-4nzhr\") pod \"network-check-target-jqk4m\" (UID: \"21463221-4b14-49d1-99d0-686a9b61c757\") " pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:34.210590 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.210500    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f7ghg"
Jul 29 21:17:34.221125 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.221100    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ql2h5\""
Jul 29 21:17:34.225444 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.225409    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-929b2\""
Jul 29 21:17:34.228267 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.228241    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8g669"
Jul 29 21:17:34.229249 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.229225    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-q4csj"]
Jul 29 21:17:34.232717 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:34.232657    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416db6ee_0be2_4e75_95c7_bd3d2da60d8a.slice/crio-37a8a0eeab200db989d3366e4c27a52e4494e45ef58fd5063dc942588ecff056 WatchSource:0}: Error finding container 37a8a0eeab200db989d3366e4c27a52e4494e45ef58fd5063dc942588ecff056: Status 404 returned error can't find the container with id 37a8a0eeab200db989d3366e4c27a52e4494e45ef58fd5063dc942588ecff056
Jul 29 21:17:34.234129 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.233937    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:34.356389 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.355852    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f7ghg"]
Jul 29 21:17:34.358572 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:34.358547    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3921626f_33f3_4c92_bdbe_c69a9ae6e54a.slice/crio-ac775c0e0cf6b7c4bb1275d6bfd18681dfbb5080419a110099b6d792d16ec7f4 WatchSource:0}: Error finding container ac775c0e0cf6b7c4bb1275d6bfd18681dfbb5080419a110099b6d792d16ec7f4: Status 404 returned error can't find the container with id ac775c0e0cf6b7c4bb1275d6bfd18681dfbb5080419a110099b6d792d16ec7f4
Jul 29 21:17:34.583006 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.582936    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqk4m"]
Jul 29 21:17:34.585936 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:34.585905    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21463221_4b14_49d1_99d0_686a9b61c757.slice/crio-f39b06550c8d14b3371258bf69cf4212ebe623d86ac1c56e0b2b005aca16c547 WatchSource:0}: Error finding container f39b06550c8d14b3371258bf69cf4212ebe623d86ac1c56e0b2b005aca16c547: Status 404 returned error can't find the container with id f39b06550c8d14b3371258bf69cf4212ebe623d86ac1c56e0b2b005aca16c547
Jul 29 21:17:34.588581 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.588502    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8g669"]
Jul 29 21:17:34.682352 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.682308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj" event={"ID":"416db6ee-0be2-4e75-95c7-bd3d2da60d8a","Type":"ContainerStarted","Data":"37a8a0eeab200db989d3366e4c27a52e4494e45ef58fd5063dc942588ecff056"}
Jul 29 21:17:34.683331 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.683298    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f7ghg" event={"ID":"3921626f-33f3-4c92-bdbe-c69a9ae6e54a","Type":"ContainerStarted","Data":"ac775c0e0cf6b7c4bb1275d6bfd18681dfbb5080419a110099b6d792d16ec7f4"}
Jul 29 21:17:34.684639 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.684616    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmmvr" event={"ID":"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04","Type":"ContainerStarted","Data":"aa9373102ebafc8c055ee0531a2cb02dd7874986e04634cd032a93a88fbc8f15"}
Jul 29 21:17:34.685610 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.685582    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqk4m" event={"ID":"21463221-4b14-49d1-99d0-686a9b61c757","Type":"ContainerStarted","Data":"f39b06550c8d14b3371258bf69cf4212ebe623d86ac1c56e0b2b005aca16c547"}
Jul 29 21:17:34.686503 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:34.686484    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8g669" event={"ID":"46824038-d39b-48cd-9bd4-32177f2500f2","Type":"ContainerStarted","Data":"1a551482209b4b07fcfbc463582e36ab8a3ffeaee85eef801953643392b0a30f"}
Jul 29 21:17:35.314131 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:35.314094    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:35.389976 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:35.389939    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:35.468948 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:35.468909    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:35.589672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:35.589587    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:35.771285 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:35.771209    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:35.833187 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:35.833129    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.206224 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.206082    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.277657 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.277591    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.382471 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.382416    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.460392 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.460313    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:36.583419 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.583378    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:36.587108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.587071    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/956e2318-8d5c-46db-9e95-5f0ff311393b-metrics-tls\") pod \"dns-default-m6l9g\" (UID: \"956e2318-8d5c-46db-9e95-5f0ff311393b\") " pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:36.812611 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.812510    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6msdl\""
Jul 29 21:17:36.820502 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.820466    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:36.983929 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:36.983892    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.175669 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:37.175582    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.629919 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:37.629877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.871563 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:37.871468    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:37.967943 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:37.967907    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:38.580104 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.580072    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"]
Jul 29 21:17:38.611617 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.611578    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"]
Jul 29 21:17:38.611770 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.611717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.615600 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.615201    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 29 21:17:38.615600 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.615328    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 29 21:17:38.615600 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.615434    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 29 21:17:38.615831 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.615645    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-c64vb\""
Jul 29 21:17:38.615970 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.615951    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 29 21:17:38.625633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.625611    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"]
Jul 29 21:17:38.625753 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.625653    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"]
Jul 29 21:17:38.625753 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.625742    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.631518 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.631495    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 21:17:38.631856 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.631603    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 21:17:38.632853 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.632831    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 21:17:38.632964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.632836    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8rgtl\""
Jul 29 21:17:38.641523 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.641488    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 21:17:38.651267 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.651245    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"]
Jul 29 21:17:38.651476 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:38.651451    2572 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-lbhrf registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-lbhrf registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4" podUID="30282750-6f76-4a07-a7ce-789832f0b914"
Jul 29 21:17:38.691455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.691416    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bbd458499-cw25b"]
Jul 29 21:17:38.702656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702628    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-registry-certificates\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.702809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702678    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-installation-pull-secrets\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.702809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702745    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-image-registry-private-configuration\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.702809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702777    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-bound-sa-token\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.702809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702801    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30282750-6f76-4a07-a7ce-789832f0b914-ca-trust-extracted\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.703061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702823    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhrf\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-kube-api-access-lbhrf\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.703061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.702966    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e495f597-f8e4-4bd1-9338-5e389af84c44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-dd7f44f6d-qmbls\" (UID: \"e495f597-f8e4-4bd1-9338-5e389af84c44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.703061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.703009    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-registry-tls\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.703061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.703037    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-trusted-ca\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.703281 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.703067    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv7vf\" (UniqueName: \"kubernetes.io/projected/e495f597-f8e4-4bd1-9338-5e389af84c44-kube-api-access-gv7vf\") pod \"managed-serviceaccount-addon-agent-dd7f44f6d-qmbls\" (UID: \"e495f597-f8e4-4bd1-9338-5e389af84c44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.705568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.705550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.706028 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.706009    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.716481 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.716461    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bbd458499-cw25b"]
Jul 29 21:17:38.723599 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.723572    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.803591 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803501    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d374fdb0-fba8-4a86-8516-489a5c1249a0-installation-pull-secrets\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.803591 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803575    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-registry-tls\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.803802 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803606    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-bound-sa-token\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.803802 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803635    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e495f597-f8e4-4bd1-9338-5e389af84c44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-dd7f44f6d-qmbls\" (UID: \"e495f597-f8e4-4bd1-9338-5e389af84c44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.803802 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d374fdb0-fba8-4a86-8516-489a5c1249a0-ca-trust-extracted\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.803802 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803684    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d374fdb0-fba8-4a86-8516-489a5c1249a0-registry-certificates\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.804002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803828    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d374fdb0-fba8-4a86-8516-489a5c1249a0-image-registry-private-configuration\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.804002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803887    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-registry-tls\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803914    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-trusted-ca\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803943    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv7vf\" (UniqueName: \"kubernetes.io/projected/e495f597-f8e4-4bd1-9338-5e389af84c44-kube-api-access-gv7vf\") pod \"managed-serviceaccount-addon-agent-dd7f44f6d-qmbls\" (UID: \"e495f597-f8e4-4bd1-9338-5e389af84c44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.804002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.803978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-registry-certificates\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804277 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804017    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-installation-pull-secrets\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804277 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804048    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5kwh\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-kube-api-access-d5kwh\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.804277 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804070    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-image-registry-private-configuration\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804277 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804100    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-bound-sa-token\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804277 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804151    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30282750-6f76-4a07-a7ce-789832f0b914-ca-trust-extracted\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804277 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804202    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhrf\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-kube-api-access-lbhrf\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.804568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804277    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d374fdb0-fba8-4a86-8516-489a5c1249a0-trusted-ca\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.804788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30282750-6f76-4a07-a7ce-789832f0b914-ca-trust-extracted\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.805019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.804995    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-registry-certificates\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.805377 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.805331    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-trusted-ca\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.806920 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.806900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-image-registry-private-configuration\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.807023 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.806930    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-registry-tls\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.807303 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.807253    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-installation-pull-secrets\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.807303 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.807271    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e495f597-f8e4-4bd1-9338-5e389af84c44-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-dd7f44f6d-qmbls\" (UID: \"e495f597-f8e4-4bd1-9338-5e389af84c44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.817299 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.817271    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-bound-sa-token\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.817624 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.817607    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv7vf\" (UniqueName: \"kubernetes.io/projected/e495f597-f8e4-4bd1-9338-5e389af84c44-kube-api-access-gv7vf\") pod \"managed-serviceaccount-addon-agent-dd7f44f6d-qmbls\" (UID: \"e495f597-f8e4-4bd1-9338-5e389af84c44\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.822821 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.822801    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhrf\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-kube-api-access-lbhrf\") pod \"image-registry-7c7dbdfc5b-cwkw4\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") " pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:38.856261 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.856226    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:38.905162 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905130    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-bound-sa-token\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905192    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-installation-pull-secrets\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905226    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-image-registry-private-configuration\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905252    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30282750-6f76-4a07-a7ce-789832f0b914-ca-trust-extracted\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905287    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbhrf\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-kube-api-access-lbhrf\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905316    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-trusted-ca\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905356    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-registry-tls\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905404    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-registry-certificates\") pod \"30282750-6f76-4a07-a7ce-789832f0b914\" (UID: \"30282750-6f76-4a07-a7ce-789832f0b914\") "
Jul 29 21:17:38.905622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905490    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d374fdb0-fba8-4a86-8516-489a5c1249a0-installation-pull-secrets\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905523    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:38.905622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905551    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-registry-tls\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905578    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-bound-sa-token\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905611    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d374fdb0-fba8-4a86-8516-489a5c1249a0-ca-trust-extracted\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905635    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d374fdb0-fba8-4a86-8516-489a5c1249a0-registry-certificates\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905673    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d374fdb0-fba8-4a86-8516-489a5c1249a0-image-registry-private-configuration\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905749    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5kwh\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-kube-api-access-d5kwh\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.905903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905797    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d374fdb0-fba8-4a86-8516-489a5c1249a0-trusted-ca\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.907214 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.906867    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d374fdb0-fba8-4a86-8516-489a5c1249a0-trusted-ca\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.908356 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.907718    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d374fdb0-fba8-4a86-8516-489a5c1249a0-registry-certificates\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.908356 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905634    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30282750-6f76-4a07-a7ce-789832f0b914-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:17:38.908356 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.905978    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:17:38.908356 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.906443    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:17:38.908356 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.908308    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d374fdb0-fba8-4a86-8516-489a5c1249a0-ca-trust-extracted\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.908685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.908616    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:17:38.909563 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.909524    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-kube-api-access-lbhrf" (OuterVolumeSpecName: "kube-api-access-lbhrf") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "kube-api-access-lbhrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:17:38.910150 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.910115    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:17:38.910478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.910453    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d374fdb0-fba8-4a86-8516-489a5c1249a0-installation-pull-secrets\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.910573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.910453    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:17:38.910573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.910500    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/45e75ef5-8b16-4b9e-9715-d50c47da09f0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-8vhcv\" (UID: \"45e75ef5-8b16-4b9e-9715-d50c47da09f0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:38.910573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.910541    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "30282750-6f76-4a07-a7ce-789832f0b914" (UID: "30282750-6f76-4a07-a7ce-789832f0b914"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:17:38.910962 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.910941    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d374fdb0-fba8-4a86-8516-489a5c1249a0-image-registry-private-configuration\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.912084 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.912066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-registry-tls\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.923865 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.923842    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5kwh\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-kube-api-access-d5kwh\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.924772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.924750    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d374fdb0-fba8-4a86-8516-489a5c1249a0-bound-sa-token\") pod \"image-registry-5bbd458499-cw25b\" (UID: \"d374fdb0-fba8-4a86-8516-489a5c1249a0\") " pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:38.933790 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.933771    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"
Jul 29 21:17:38.952712 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:38.952672    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:39.007107 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007073    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-registry-tls\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007107 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007108    2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-registry-certificates\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007125    2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-bound-sa-token\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007141    2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-installation-pull-secrets\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007156    2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30282750-6f76-4a07-a7ce-789832f0b914-image-registry-private-configuration\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007184    2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30282750-6f76-4a07-a7ce-789832f0b914-ca-trust-extracted\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007199    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbhrf\" (UniqueName: \"kubernetes.io/projected/30282750-6f76-4a07-a7ce-789832f0b914-kube-api-access-lbhrf\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.007353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.007213    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30282750-6f76-4a07-a7ce-789832f0b914-trusted-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:17:39.020061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.020032    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:39.069278 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.069200    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:39.147881 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.147855    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-g5w5t\""
Jul 29 21:17:39.156798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.156775    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"
Jul 29 21:17:39.313184 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:39.313127    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30282750_6f76_4a07_a7ce_789832f0b914.slice\": RecentStats: unable to find data in memory cache]"
Jul 29 21:17:39.512477 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.512439    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:39.713406 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.711398    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"
Jul 29 21:17:39.753743 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.753706    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"]
Jul 29 21:17:39.755532 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.755496    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c7dbdfc5b-cwkw4"]
Jul 29 21:17:39.771409 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.771328    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:39.839361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:39.839319    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:41.303019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.302985    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30282750-6f76-4a07-a7ce-789832f0b914" path="/var/lib/kubelet/pods/30282750-6f76-4a07-a7ce-789832f0b914/volumes"
Jul 29 21:17:41.728407 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.728326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj" event={"ID":"416db6ee-0be2-4e75-95c7-bd3d2da60d8a","Type":"ContainerStarted","Data":"ca24e3f655809c9a69efabb73c2a0abf6c4bac2ceb6fa3b74e0e6f987e10f17f"}
Jul 29 21:17:41.732720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.732652    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmmvr" event={"ID":"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04","Type":"ContainerStarted","Data":"ff87aacda34bb118368e72728bccc36e67ccf6a06e2e460e3d39b248f45ddf47"}
Jul 29 21:17:41.735756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.735730    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqk4m" event={"ID":"21463221-4b14-49d1-99d0-686a9b61c757","Type":"ContainerStarted","Data":"34bb1a1cd50239b0a8dd44d2f7b0edda2bbac9ec83af3f4a1e250f54f1716e2d"}
Jul 29 21:17:41.735924 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.735909    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:17:41.756263 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.755792    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-q4csj" podStartSLOduration=20.130020354 podStartE2EDuration="26.75576975s" totalImagesPullingTime="6.625749396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:41.747042742 +0000 UTC m=+72.963820205" watchObservedRunningTime="2026-07-29 21:17:41.75576975 +0000 UTC m=+72.972547213"
Jul 29 21:17:41.759877 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.759775    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv"]
Jul 29 21:17:41.776506 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.775728    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jqk4m" podStartSLOduration=65.833310851 podStartE2EDuration="1m12.775709868s" totalImagesPullingTime="6.942399017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:41.775567498 +0000 UTC m=+72.992344960" watchObservedRunningTime="2026-07-29 21:17:41.775709868 +0000 UTC m=+72.992487330"
Jul 29 21:17:41.792601 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.792506    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m6l9g"]
Jul 29 21:17:41.799959 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:41.799912    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956e2318_8d5c_46db_9e95_5f0ff311393b.slice/crio-6e4825f081b69754b1a361912b03f7e58cc8ba08d89da8f708235da3185810db WatchSource:0}: Error finding container 6e4825f081b69754b1a361912b03f7e58cc8ba08d89da8f708235da3185810db: Status 404 returned error can't find the container with id 6e4825f081b69754b1a361912b03f7e58cc8ba08d89da8f708235da3185810db
Jul 29 21:17:41.800222 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.800143    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls"]
Jul 29 21:17:41.804464 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:41.804390    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode495f597_f8e4_4bd1_9338_5e389af84c44.slice/crio-cc432606c571c0adbf9d2dfa491a4c9fb55522a84c89a8f2f1f7f8db977947aa WatchSource:0}: Error finding container cc432606c571c0adbf9d2dfa491a4c9fb55522a84c89a8f2f1f7f8db977947aa: Status 404 returned error can't find the container with id cc432606c571c0adbf9d2dfa491a4c9fb55522a84c89a8f2f1f7f8db977947aa
Jul 29 21:17:41.818772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.818729    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bbd458499-cw25b"]
Jul 29 21:17:41.928676 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:41.928634    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:42.180765 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.180710    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:42.746005 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.745966    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8g669" event={"ID":"46824038-d39b-48cd-9bd4-32177f2500f2","Type":"ContainerStarted","Data":"00007d5037438f21367ef72c62451e7b78e68b96bbd19038a5d70d7fb81930a3"}
Jul 29 21:17:42.746487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.746033    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8g669" event={"ID":"46824038-d39b-48cd-9bd4-32177f2500f2","Type":"ContainerStarted","Data":"602a12aededa3f04a811771491d152c2de9378c79b607ecb8464e6f5a6959a20"}
Jul 29 21:17:42.747316 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.747290    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls" event={"ID":"e495f597-f8e4-4bd1-9338-5e389af84c44","Type":"ContainerStarted","Data":"cc432606c571c0adbf9d2dfa491a4c9fb55522a84c89a8f2f1f7f8db977947aa"}
Jul 29 21:17:42.748664 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.748600    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv" event={"ID":"45e75ef5-8b16-4b9e-9715-d50c47da09f0","Type":"ContainerStarted","Data":"b911ab3f4df19fb91a13bf7ee8f40ea77b481ba2f241f5ed324c323ca1b1743e"}
Jul 29 21:17:42.750478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.750452    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m6l9g" event={"ID":"956e2318-8d5c-46db-9e95-5f0ff311393b","Type":"ContainerStarted","Data":"6e4825f081b69754b1a361912b03f7e58cc8ba08d89da8f708235da3185810db"}
Jul 29 21:17:42.752467 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.752441    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f7ghg" event={"ID":"3921626f-33f3-4c92-bdbe-c69a9ae6e54a","Type":"ContainerStarted","Data":"020680fb39b3d1958e2455ad4d7c82ed7553eabb432048df23345eeef6314b2a"}
Jul 29 21:17:42.754819 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.754743    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bbd458499-cw25b" event={"ID":"d374fdb0-fba8-4a86-8516-489a5c1249a0","Type":"ContainerStarted","Data":"746fc2988f02f5a4058b10a23957a74bb0c9a4b3e903bbc913522e98d464f26d"}
Jul 29 21:17:42.754819 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.754783    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bbd458499-cw25b" event={"ID":"d374fdb0-fba8-4a86-8516-489a5c1249a0","Type":"ContainerStarted","Data":"fedefa4711dfde82999e6579f6b6ceed6d0f30407c81e9c45db029ee0adfa40e"}
Jul 29 21:17:42.754988 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.754827    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:17:42.770133 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.770088    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8g669" podStartSLOduration=66.845178803 podStartE2EDuration="1m13.770068479s" totalImagesPullingTime="6.924889676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:42.764557609 +0000 UTC m=+73.981335072" watchObservedRunningTime="2026-07-29 21:17:42.770068479 +0000 UTC m=+73.986845936"
Jul 29 21:17:42.781825 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.781784    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f7ghg" podStartSLOduration=66.582891063 podStartE2EDuration="1m13.781766328s" totalImagesPullingTime="7.198875265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:42.781220022 +0000 UTC m=+73.997997485" watchObservedRunningTime="2026-07-29 21:17:42.781766328 +0000 UTC m=+73.998543771"
Jul 29 21:17:42.864363 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:42.864310    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:43.060842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:43.060729    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:48.168862 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.168770    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:48.271717 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.271678    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:48.353681 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.353639    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:48.360070 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.360010    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bbd458499-cw25b" podStartSLOduration=10.359988761 podStartE2EDuration="10.359988761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:42.801594299 +0000 UTC m=+74.018371763" watchObservedRunningTime="2026-07-29 21:17:48.359988761 +0000 UTC m=+79.576766204"
Jul 29 21:17:48.360410 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.360393    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"]
Jul 29 21:17:48.362931 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.362913    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:48.365236 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.365219    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 21:17:48.365339 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.365255    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rsjmj\""
Jul 29 21:17:48.372663 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.372635    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"]
Jul 29 21:17:48.489514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.489477    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee7b230f-8685-4ca2-b382-fdb37b89ed14-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xkmh9\" (UID: \"ee7b230f-8685-4ca2-b382-fdb37b89ed14\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:48.562514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.562479    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:48.590892 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.590863    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee7b230f-8685-4ca2-b382-fdb37b89ed14-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xkmh9\" (UID: \"ee7b230f-8685-4ca2-b382-fdb37b89ed14\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:48.593334 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.593310    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee7b230f-8685-4ca2-b382-fdb37b89ed14-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-xkmh9\" (UID: \"ee7b230f-8685-4ca2-b382-fdb37b89ed14\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:48.672564 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.672511    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:48.783373 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.783320    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls" event={"ID":"e495f597-f8e4-4bd1-9338-5e389af84c44","Type":"ContainerStarted","Data":"e83455aea0102ecab8a7a376ff4f5f3794b001f0c2d1db73d145c5e6f0df599c"}
Jul 29 21:17:48.785303 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.785277    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv" event={"ID":"45e75ef5-8b16-4b9e-9715-d50c47da09f0","Type":"ContainerStarted","Data":"b825d9c0d3e1541705b1d669e7caf5378928c22f4dd77cf71228ed4111f46155"}
Jul 29 21:17:48.787235 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.787205    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m6l9g" event={"ID":"956e2318-8d5c-46db-9e95-5f0ff311393b","Type":"ContainerStarted","Data":"e2d7f3db79b5ba0e4faa89dbb4284f130faef753586130bb040521490ce15e33"}
Jul 29 21:17:48.787391 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.787258    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m6l9g" event={"ID":"956e2318-8d5c-46db-9e95-5f0ff311393b","Type":"ContainerStarted","Data":"db28bffbb8b4a68afe9fae5f0cdd29484f7f412d68e2ab253576b8349c6f0121"}
Jul 29 21:17:48.787391 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.787297    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:48.789201 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.789160    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mmmvr" event={"ID":"b6dc41d9-5eaa-482c-822a-b9e18c2f6b04","Type":"ContainerStarted","Data":"48d5894e0361bba30eaca6d52bf7e78dbabd0bffc0d3756cdd50c644694841b0"}
Jul 29 21:17:48.801969 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.801926    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-dd7f44f6d-qmbls" podStartSLOduration=4.862694104 podStartE2EDuration="10.801912891s" totalImagesPullingTime="5.939218787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:48.80078871 +0000 UTC m=+80.017566173" watchObservedRunningTime="2026-07-29 21:17:48.801912891 +0000 UTC m=+80.018690656"
Jul 29 21:17:48.803637 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.803619    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"]
Jul 29 21:17:48.805372 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:48.805350    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7b230f_8685_4ca2_b382_fdb37b89ed14.slice/crio-a90b2a01dca425f832a448c6098104d90f004dd9a175791c5d174d7d182d7ec1 WatchSource:0}: Error finding container a90b2a01dca425f832a448c6098104d90f004dd9a175791c5d174d7d182d7ec1: Status 404 returned error can't find the container with id a90b2a01dca425f832a448c6098104d90f004dd9a175791c5d174d7d182d7ec1
Jul 29 21:17:48.820255 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.820200    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-8vhcv" podStartSLOduration=36.851527673 podStartE2EDuration="42.820156641s" totalImagesPullingTime="5.968628968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:48.819216112 +0000 UTC m=+80.035993576" watchObservedRunningTime="2026-07-29 21:17:48.820156641 +0000 UTC m=+80.036934104"
Jul 29 21:17:48.839045 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.838990    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m6l9g" podStartSLOduration=38.900257595 podStartE2EDuration="44.838974305s" totalImagesPullingTime="5.93871671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:48.838656362 +0000 UTC m=+80.055433827" watchObservedRunningTime="2026-07-29 21:17:48.838974305 +0000 UTC m=+80.055751766"
Jul 29 21:17:48.858988 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:48.858922    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mmmvr" podStartSLOduration=18.214215397 podStartE2EDuration="36.858898094s" totalImagesPullingTime="18.644682697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 21:17:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:48.857338121 +0000 UTC m=+80.074115609" watchObservedRunningTime="2026-07-29 21:17:48.858898094 +0000 UTC m=+80.075675557"
Jul 29 21:17:49.800101 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:49.800062    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9" event={"ID":"ee7b230f-8685-4ca2-b382-fdb37b89ed14","Type":"ContainerStarted","Data":"a90b2a01dca425f832a448c6098104d90f004dd9a175791c5d174d7d182d7ec1"}
Jul 29 21:17:50.804626 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:50.804587    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9" event={"ID":"ee7b230f-8685-4ca2-b382-fdb37b89ed14","Type":"ContainerStarted","Data":"667482e0ae30d8c4002febf2d0868f9d8a1991343778f69cdc6f73bcb0a25027"}
Jul 29 21:17:50.805047 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:50.804802    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:50.809409 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:50.809386    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9"
Jul 29 21:17:50.820301 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:50.820259    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-xkmh9" podStartSLOduration=1.622967813 podStartE2EDuration="2.820243775s" totalImagesPullingTime="1.197275962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:50.819022647 +0000 UTC m=+82.035800111" watchObservedRunningTime="2026-07-29 21:17:50.820243775 +0000 UTC m=+82.037021237"
Jul 29 21:17:50.999479 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:50.999442    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:51.061749 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.061659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:51.398416 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.398326    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:51.429605 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.429572    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"]
Jul 29 21:17:51.433939 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.433923    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.437227 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.437201    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 21:17:51.437366 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.437202    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k79xt\""
Jul 29 21:17:51.437366 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.437202    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 29 21:17:51.437366 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.437208    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 29 21:17:51.444480 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.444456    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"]
Jul 29 21:17:51.471077 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.471042    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:51.514285 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.514151    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.514285 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.514246    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.514285 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.514284    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pg7k\" (UniqueName: \"kubernetes.io/projected/63db8829-79ba-4be0-9190-1ffeff599c2e-kube-api-access-6pg7k\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.514551 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.514305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63db8829-79ba-4be0-9190-1ffeff599c2e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.615216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.615157    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.615343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.615263    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.615343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.615303    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pg7k\" (UniqueName: \"kubernetes.io/projected/63db8829-79ba-4be0-9190-1ffeff599c2e-kube-api-access-6pg7k\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.615343 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:51.615310    2572 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 29 21:17:51.615343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.615330    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63db8829-79ba-4be0-9190-1ffeff599c2e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.615528 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:17:51.615381    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-tls podName:63db8829-79ba-4be0-9190-1ffeff599c2e nodeName:}" failed. No retries permitted until 2026-07-29 21:17:52.11535863 +0000 UTC m=+83.332136070 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-gwxwx" (UID: "63db8829-79ba-4be0-9190-1ffeff599c2e") : secret "prometheus-operator-tls" not found
Jul 29 21:17:51.616091 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.616071    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63db8829-79ba-4be0-9190-1ffeff599c2e-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.617723 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.617696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.626347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.626324    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pg7k\" (UniqueName: \"kubernetes.io/projected/63db8829-79ba-4be0-9190-1ffeff599c2e-kube-api-access-6pg7k\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:51.774570 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:51.774505    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:52.120504 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:52.120392    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:52.122804 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:52.122782    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/63db8829-79ba-4be0-9190-1ffeff599c2e-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-gwxwx\" (UID: \"63db8829-79ba-4be0-9190-1ffeff599c2e\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:52.343465 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:52.343423    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"
Jul 29 21:17:52.469326 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:52.469299    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-gwxwx"]
Jul 29 21:17:52.472043 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:52.472017    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63db8829_79ba_4be0_9190_1ffeff599c2e.slice/crio-11cd78dbff1050b609617c546eb4363d6d6b99cb3013f3cd04eb7771e93ff26d WatchSource:0}: Error finding container 11cd78dbff1050b609617c546eb4363d6d6b99cb3013f3cd04eb7771e93ff26d: Status 404 returned error can't find the container with id 11cd78dbff1050b609617c546eb4363d6d6b99cb3013f3cd04eb7771e93ff26d
Jul 29 21:17:52.812807 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:52.812773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx" event={"ID":"63db8829-79ba-4be0-9190-1ffeff599c2e","Type":"ContainerStarted","Data":"11cd78dbff1050b609617c546eb4363d6d6b99cb3013f3cd04eb7771e93ff26d"}
Jul 29 21:17:53.810656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:53.810617    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:53.878587 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:53.878544    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:54.404216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:54.404090    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:54.623571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:54.623530    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:54.705365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:54.705332    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:54.821517 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:54.821481    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx" event={"ID":"63db8829-79ba-4be0-9190-1ffeff599c2e","Type":"ContainerStarted","Data":"c463d22fbba5fcdfa602d17987b7a3c94b7630243f207480bf57616f3e3585a3"}
Jul 29 21:17:54.821908 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:54.821580    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx" event={"ID":"63db8829-79ba-4be0-9190-1ffeff599c2e","Type":"ContainerStarted","Data":"a224d195df88d719a09db1566835f0adf1850dd07be21b3ef2de043be7617cc7"}
Jul 29 21:17:54.840196 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:54.840141    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-gwxwx" podStartSLOduration=2.238835886 podStartE2EDuration="3.840125297s" totalImagesPullingTime="1.601289411s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:54.838253608 +0000 UTC m=+86.055031071" watchObservedRunningTime="2026-07-29 21:17:54.840125297 +0000 UTC m=+86.056902789"
Jul 29 21:17:55.816667 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:55.816627    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:55.885256 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:55.885222    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:56.539037 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.539000    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:56.855586 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.855508    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"]
Jul 29 21:17:56.859418 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.859392    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:56.864778 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.864745    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-nnf48\""
Jul 29 21:17:56.864917 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.864824    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 29 21:17:56.874034 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.874012    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 29 21:17:56.896082 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.896039    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"]
Jul 29 21:17:56.905785 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.905755    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7fhrf"]
Jul 29 21:17:56.908954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.908931    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:56.918774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.918751    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wblq4\""
Jul 29 21:17:56.918774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.918751    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 21:17:56.919024 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.918784    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 21:17:56.919024 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.918751    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 21:17:56.925011 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.924975    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:56.954494 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.954465    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1440b53f-b05b-4f72-8747-601fbb1c0556-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:56.954494 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.954505    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1440b53f-b05b-4f72-8747-601fbb1c0556-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:56.954721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.954545    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd69m\" (UniqueName: \"kubernetes.io/projected/1440b53f-b05b-4f72-8747-601fbb1c0556-kube-api-access-cd69m\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:56.954721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:56.954601    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1440b53f-b05b-4f72-8747-601fbb1c0556-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.055180 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055147    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-tls\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055215    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd69m\" (UniqueName: \"kubernetes.io/projected/1440b53f-b05b-4f72-8747-601fbb1c0556-kube-api-access-cd69m\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.055360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055251    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-sys\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055271    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1440b53f-b05b-4f72-8747-601fbb1c0556-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.055360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-textfile\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055495 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055362    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1440b53f-b05b-4f72-8747-601fbb1c0556-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.055495 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055389    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055574 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055486    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-root\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055574 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055522    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x785g\" (UniqueName: \"kubernetes.io/projected/e18afab2-c123-4619-9e7d-e7962a72b274-kube-api-access-x785g\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055578    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1440b53f-b05b-4f72-8747-601fbb1c0556-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.055635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055605    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e18afab2-c123-4619-9e7d-e7962a72b274-metrics-client-ca\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055698 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055631    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-accelerators-collector-config\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.055698 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.055668    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-wtmp\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.056195 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.056147    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1440b53f-b05b-4f72-8747-601fbb1c0556-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.057662 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.057643    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1440b53f-b05b-4f72-8747-601fbb1c0556-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.057757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.057655    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1440b53f-b05b-4f72-8747-601fbb1c0556-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.071124 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.071092    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:57.101325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.101293    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd69m\" (UniqueName: \"kubernetes.io/projected/1440b53f-b05b-4f72-8747-601fbb1c0556-kube-api-access-cd69m\") pod \"openshift-state-metrics-b8f49f5c6-xldn5\" (UID: \"1440b53f-b05b-4f72-8747-601fbb1c0556\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.156343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156249    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x785g\" (UniqueName: \"kubernetes.io/projected/e18afab2-c123-4619-9e7d-e7962a72b274-kube-api-access-x785g\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156297    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e18afab2-c123-4619-9e7d-e7962a72b274-metrics-client-ca\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156319    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-accelerators-collector-config\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156341    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-wtmp\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156368    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-tls\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156394    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-sys\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156430    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-textfile\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156465    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156466    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-sys\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156488    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-root\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156534    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-root\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.156677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156534    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-wtmp\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.157000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.156842    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-textfile\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.157044 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.157008    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e18afab2-c123-4619-9e7d-e7962a72b274-metrics-client-ca\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.157085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.157040    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-accelerators-collector-config\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.158738 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.158713    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.158828 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.158792    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e18afab2-c123-4619-9e7d-e7962a72b274-node-exporter-tls\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.169773 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.169751    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"
Jul 29 21:17:57.183119 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.183098    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x785g\" (UniqueName: \"kubernetes.io/projected/e18afab2-c123-4619-9e7d-e7962a72b274-kube-api-access-x785g\") pod \"node-exporter-7fhrf\" (UID: \"e18afab2-c123-4619-9e7d-e7962a72b274\") " pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.218055 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.218022    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7fhrf"
Jul 29 21:17:57.232052 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:57.232010    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode18afab2_c123_4619_9e7d_e7962a72b274.slice/crio-89a6f350e424c5cea1a9c4fa0a54bb2d9e338815cc7e366ff3eb4258ab3340cc WatchSource:0}: Error finding container 89a6f350e424c5cea1a9c4fa0a54bb2d9e338815cc7e366ff3eb4258ab3340cc: Status 404 returned error can't find the container with id 89a6f350e424c5cea1a9c4fa0a54bb2d9e338815cc7e366ff3eb4258ab3340cc
Jul 29 21:17:57.314809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.314774    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5"]
Jul 29 21:17:57.317516 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:57.317488    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1440b53f_b05b_4f72_8747_601fbb1c0556.slice/crio-e77ef5c2bc28f095ac01c2367f68b34d816ea210fcb568bf038859015166fb31 WatchSource:0}: Error finding container e77ef5c2bc28f095ac01c2367f68b34d816ea210fcb568bf038859015166fb31: Status 404 returned error can't find the container with id e77ef5c2bc28f095ac01c2367f68b34d816ea210fcb568bf038859015166fb31
Jul 29 21:17:57.409842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.409774    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:57.640842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.640785    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:57.835418 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.835377    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5" event={"ID":"1440b53f-b05b-4f72-8747-601fbb1c0556","Type":"ContainerStarted","Data":"88f6219ddff886d0284755739aa44d6986f5ef356d56427a1390155730b1261a"}
Jul 29 21:17:57.835610 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.835473    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5" event={"ID":"1440b53f-b05b-4f72-8747-601fbb1c0556","Type":"ContainerStarted","Data":"69d3d8bd6dd7fccab37bf1df67f65b401564c1f4d1c5fc7433820ebdef7a8bc1"}
Jul 29 21:17:57.835610 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.835507    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5" event={"ID":"1440b53f-b05b-4f72-8747-601fbb1c0556","Type":"ContainerStarted","Data":"e77ef5c2bc28f095ac01c2367f68b34d816ea210fcb568bf038859015166fb31"}
Jul 29 21:17:57.836629 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.836596    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fhrf" event={"ID":"e18afab2-c123-4619-9e7d-e7962a72b274","Type":"ContainerStarted","Data":"89a6f350e424c5cea1a9c4fa0a54bb2d9e338815cc7e366ff3eb4258ab3340cc"}
Jul 29 21:17:57.857800 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.857759    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:57.964415 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:57.964367    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:58.078514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.078480    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:58.802903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.802871    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m6l9g"
Jul 29 21:17:58.843463 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.843437    2572 generic.go:356] "Generic (PLEG): container finished" podID="e18afab2-c123-4619-9e7d-e7962a72b274" containerID="9642cab0cfe33b87526ec7b0d36115fc5e9ac0fc1c8019fe741946b1e819572f" exitCode=0
Jul 29 21:17:58.843579 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.843497    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fhrf" event={"ID":"e18afab2-c123-4619-9e7d-e7962a72b274","Type":"ContainerDied","Data":"9642cab0cfe33b87526ec7b0d36115fc5e9ac0fc1c8019fe741946b1e819572f"}
Jul 29 21:17:58.860853 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.860830    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-55c45d54bc-w62kp"]
Jul 29 21:17:58.866199 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.866161    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.868767 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.868750    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 29 21:17:58.869038 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.869017    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 29 21:17:58.869115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.869054    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 29 21:17:58.869216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.869196    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 29 21:17:58.869284 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.869222    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-q6n7h\""
Jul 29 21:17:58.869284 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.869245    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2br9k61t97kh6\""
Jul 29 21:17:58.869615 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.869601    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 29 21:17:58.883954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.883929    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55c45d54bc-w62kp"]
Jul 29 21:17:58.971924 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.971898    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.971935    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.971960    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.972059    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-grpc-tls\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.972095    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-tls\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.972122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a866806-4c3b-4de1-9eaf-fe12e47ee327-metrics-client-ca\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972594 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.972368    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:58.972594 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:58.972400    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqgbk\" (UniqueName: \"kubernetes.io/projected/4a866806-4c3b-4de1-9eaf-fe12e47ee327-kube-api-access-hqgbk\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073058 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.072980    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073058 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073020    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqgbk\" (UniqueName: \"kubernetes.io/projected/4a866806-4c3b-4de1-9eaf-fe12e47ee327-kube-api-access-hqgbk\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073058 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073044    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073303 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073198    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073303 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073261    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073315    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-grpc-tls\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073349    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-tls\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.073401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.073379    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a866806-4c3b-4de1-9eaf-fe12e47ee327-metrics-client-ca\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.074227 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.074153    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a866806-4c3b-4de1-9eaf-fe12e47ee327-metrics-client-ca\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.075779 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.075746    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.075923 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.075900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-tls\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.076021 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.076002    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.076080 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.076034    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.076396 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.076380    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.076550 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.076531    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a866806-4c3b-4de1-9eaf-fe12e47ee327-secret-grpc-tls\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.080602 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.080584    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqgbk\" (UniqueName: \"kubernetes.io/projected/4a866806-4c3b-4de1-9eaf-fe12e47ee327-kube-api-access-hqgbk\") pod \"thanos-querier-55c45d54bc-w62kp\" (UID: \"4a866806-4c3b-4de1-9eaf-fe12e47ee327\") " pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.205667 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.205636    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:17:59.351651 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.351624    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55c45d54bc-w62kp"]
Jul 29 21:17:59.353607 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:17:59.353564    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a866806_4c3b_4de1_9eaf_fe12e47ee327.slice/crio-bbd602405cec15ac7651dd69c5cf6279af81e537dd9d48cf55ca04a037536ca1 WatchSource:0}: Error finding container bbd602405cec15ac7651dd69c5cf6279af81e537dd9d48cf55ca04a037536ca1: Status 404 returned error can't find the container with id bbd602405cec15ac7651dd69c5cf6279af81e537dd9d48cf55ca04a037536ca1
Jul 29 21:17:59.515776 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.515736    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:59.614434 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.614346    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:59.741291 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.741247    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:59.842513 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.842480    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:17:59.850004 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.849973    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fhrf" event={"ID":"e18afab2-c123-4619-9e7d-e7962a72b274","Type":"ContainerStarted","Data":"5a289411cc04f8663fb6b9d91622de9c7a2b4520b53f525a7d1128f7a2dff4e8"}
Jul 29 21:17:59.850195 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.850028    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7fhrf" event={"ID":"e18afab2-c123-4619-9e7d-e7962a72b274","Type":"ContainerStarted","Data":"e52807d9503d00f970f669c55a815ecff692bd723a3443d5a0612ca8a8490da0"}
Jul 29 21:17:59.851286 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.851257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"bbd602405cec15ac7651dd69c5cf6279af81e537dd9d48cf55ca04a037536ca1"}
Jul 29 21:17:59.853148 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.853124    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5" event={"ID":"1440b53f-b05b-4f72-8747-601fbb1c0556","Type":"ContainerStarted","Data":"ab63cda1a97d619d1392198665bff6750e898295faf1a6b335156b224dd2afb9"}
Jul 29 21:17:59.873964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.873884    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7fhrf" podStartSLOduration=2.847731991 podStartE2EDuration="3.873867598s" totalImagesPullingTime="960.711092ms" totalInitContainerRuntime="65.424515ms" isStatefulPod=true podCreationTimestamp="2026-07-29 21:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:59.872495859 +0000 UTC m=+91.089273323" watchObservedRunningTime="2026-07-29 21:17:59.873867598 +0000 UTC m=+91.090645060"
Jul 29 21:17:59.910348 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.910298    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-xldn5" podStartSLOduration=2.520810703 podStartE2EDuration="3.910281703s" totalImagesPullingTime="1.389471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:17:59.909443832 +0000 UTC m=+91.126221295" watchObservedRunningTime="2026-07-29 21:17:59.910281703 +0000 UTC m=+91.127059164"
Jul 29 21:17:59.987778 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:17:59.987741    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:00.068294 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:00.068255    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:01.622818 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:01.622770    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:01.738952 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:01.738913    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:01.922270 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:01.922238    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:02.401601 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:02.401558    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:02.510749 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:02.510710    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:02.867360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:02.867298    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"02690072b4787979d0834f931933c98c65edf2573adb7a9ae78bb3e9ec792cc0"}
Jul 29 21:18:02.867776 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:02.867426    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"9571efd68152ee7d6b9443ce7163f92ffffedacc95071ca6114c624981616cf9"}
Jul 29 21:18:02.867776 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:02.867471    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"86611374a82daeb62f0eb36a6548f60fe3c5507cb58e86e24957335e0591a36b"}
Jul 29 21:18:03.106067 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:03.106022    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:03.765490 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:03.764991    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bbd458499-cw25b"
Jul 29 21:18:03.875443 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:03.875406    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"7d41785481e559d73b862a0292842452bce80082be94d33796871dbc5e8d431c"}
Jul 29 21:18:03.957660 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:03.957253    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:04.056422 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:04.056386    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:04.882388 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:04.882349    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"84a486e55f88829f30b6a68427e38507f3ebad9c0c99e26b15c6f884a147b743"}
Jul 29 21:18:04.882885 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:04.882416    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" event={"ID":"4a866806-4c3b-4de1-9eaf-fe12e47ee327","Type":"ContainerStarted","Data":"fd9b6f245e4b089edeffcc6277a2433561279ab9baf2c20b719af2ed81cb263a"}
Jul 29 21:18:04.882885 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:04.882517    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:18:04.913858 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:04.913809    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp" podStartSLOduration=2.75558719 podStartE2EDuration="6.913791873s" totalImagesPullingTime="4.158204683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:17:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:18:04.912344306 +0000 UTC m=+96.129121770" watchObservedRunningTime="2026-07-29 21:18:04.913791873 +0000 UTC m=+96.130569332"
Jul 29 21:18:05.785584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:05.785546    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:05.856330 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:05.856290    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:07.424198 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:07.424153    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:07.524492 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:07.524455    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.504476 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:08.504406    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.605650 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:08.605619    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:08.677610 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:08.677573    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:10.892386 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:10.892355    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-55c45d54bc-w62kp"
Jul 29 21:18:11.269895 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:11.269854    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:11.344972 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:11.344933    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:12.757972 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:12.757941    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jqk4m"
Jul 29 21:18:12.812382 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:12.812346    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:12.967912 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:12.967867    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:17.835993 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:17.835952    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:17.934578 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:17.933565    2572 generic.go:356] "Generic (PLEG): container finished" podID="b9ebc0f6-62d4-4f59-aea4-674376a9dab8" containerID="cc502841e8b8761f5420a373ca1fa3fc7b3289bcf0366646d8d530e59fa4cba4" exitCode=0
Jul 29 21:18:17.934578 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:17.933630    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542" event={"ID":"b9ebc0f6-62d4-4f59-aea4-674376a9dab8","Type":"ContainerDied","Data":"cc502841e8b8761f5420a373ca1fa3fc7b3289bcf0366646d8d530e59fa4cba4"}
Jul 29 21:18:17.934578 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:17.934017    2572 scope.go:117] "RemoveContainer" containerID="cc502841e8b8761f5420a373ca1fa3fc7b3289bcf0366646d8d530e59fa4cba4"
Jul 29 21:18:17.979682 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:17.979647    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:18.941887 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:18.941856    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-6x542" event={"ID":"b9ebc0f6-62d4-4f59-aea4-674376a9dab8","Type":"ContainerStarted","Data":"52413052c49756950567b6751b893add2f4eebd9d7b7206043c9c9bda433ad2c"}
Jul 29 21:18:19.455926 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:19.455892    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:19.562128 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:19.562095    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:24.594483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:24.594440    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:24.658789 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:24.658743    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:27.977852 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:27.977812    2572 generic.go:356] "Generic (PLEG): container finished" podID="ddf60a13-3a1c-41fa-8212-6762ab9a37c6" containerID="042d4b5d8147811b0060d990574e5eae21cddbbee349b1956a54a03290a1d6a4" exitCode=0
Jul 29 21:18:27.978405 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:27.977886    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zd96v" event={"ID":"ddf60a13-3a1c-41fa-8212-6762ab9a37c6","Type":"ContainerDied","Data":"042d4b5d8147811b0060d990574e5eae21cddbbee349b1956a54a03290a1d6a4"}
Jul 29 21:18:27.978405 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:27.978249    2572 scope.go:117] "RemoveContainer" containerID="042d4b5d8147811b0060d990574e5eae21cddbbee349b1956a54a03290a1d6a4"
Jul 29 21:18:28.984033 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:28.983993    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-zd96v" event={"ID":"ddf60a13-3a1c-41fa-8212-6762ab9a37c6","Type":"ContainerStarted","Data":"91d25df5d343c6b434021265f3096344d0594f8f83b0a5665a78532378e2e0ee"}
Jul 29 21:18:39.592896 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:39.592861    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:39.656260 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:39.656226    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:41.097711 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:41.097670    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:41.199282 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:41.199136    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:18:43.039441 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:43.039406    2572 generic.go:356] "Generic (PLEG): container finished" podID="1deada3b-1099-4beb-b51f-b4839686a1a8" containerID="9199059e363646cd870b24c3f2924bd83f57b233db24e3a01928aca14a5adbf5" exitCode=0
Jul 29 21:18:43.039870 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:43.039476    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl" event={"ID":"1deada3b-1099-4beb-b51f-b4839686a1a8","Type":"ContainerDied","Data":"9199059e363646cd870b24c3f2924bd83f57b233db24e3a01928aca14a5adbf5"}
Jul 29 21:18:43.039870 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:43.039782    2572 scope.go:117] "RemoveContainer" containerID="9199059e363646cd870b24c3f2924bd83f57b233db24e3a01928aca14a5adbf5"
Jul 29 21:18:44.045349 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:18:44.045311    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl" event={"ID":"1deada3b-1099-4beb-b51f-b4839686a1a8","Type":"ContainerStarted","Data":"891cb5690010324f6108cf6ed77599a08f59f0cee7fcf22c33c9f6bd0ab2c5bb"}
Jul 29 21:19:14.660047 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:14.660014    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:16.055962 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:16.055891    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:16.156868 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:16.156820    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:16.733956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:16.733913    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:26.159233 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:26.159136    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:36.982230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:36.982192    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:37.082132 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:37.082094    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:37.183288 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:37.183253    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:19:58.788610 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:19:58.788579    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:31.368898 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:31.368849    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:31.445037 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:31.444993    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:35.406780 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:35.406737    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:35.507111 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:35.507075    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:43.255254 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:43.255212    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:43.319714 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:43.319679    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:46.189256 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:46.189220    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:46.289704 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:46.289665    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:46.437351 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:46.437319    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:54.649215 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:54.649116    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:20:54.748444 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:20:54.748408    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:13.192813 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:21:13.192777    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:29.162705 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:21:29.162677    2572 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 21:21:34.319080 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:21:34.319043    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:21:36.071230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:21:36.071188    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:01.410298 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:01.410261    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:01.790898 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:01.790863    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:03.498022 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.497988    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"]
Jul 29 21:22:03.501265 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.501246    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.504603 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.504576    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7jkwv\""
Jul 29 21:22:03.504911 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.504894    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 21:22:03.505149 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.505135    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 21:22:03.515952 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.515924    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"]
Jul 29 21:22:03.553319 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.553285    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dn9v\" (UniqueName: \"kubernetes.io/projected/b84e8424-a477-4ae5-88f2-f642527563bf-kube-api-access-6dn9v\") pod \"cert-manager-operator-controller-manager-55c444bf86-ghflw\" (UID: \"b84e8424-a477-4ae5-88f2-f642527563bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.553483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.553371    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b84e8424-a477-4ae5-88f2-f642527563bf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ghflw\" (UID: \"b84e8424-a477-4ae5-88f2-f642527563bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.654292 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.654255    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dn9v\" (UniqueName: \"kubernetes.io/projected/b84e8424-a477-4ae5-88f2-f642527563bf-kube-api-access-6dn9v\") pod \"cert-manager-operator-controller-manager-55c444bf86-ghflw\" (UID: \"b84e8424-a477-4ae5-88f2-f642527563bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.654501 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.654378    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b84e8424-a477-4ae5-88f2-f642527563bf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ghflw\" (UID: \"b84e8424-a477-4ae5-88f2-f642527563bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.654756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.654734    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b84e8424-a477-4ae5-88f2-f642527563bf-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-ghflw\" (UID: \"b84e8424-a477-4ae5-88f2-f642527563bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.670216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.670182    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dn9v\" (UniqueName: \"kubernetes.io/projected/b84e8424-a477-4ae5-88f2-f642527563bf-kube-api-access-6dn9v\") pod \"cert-manager-operator-controller-manager-55c444bf86-ghflw\" (UID: \"b84e8424-a477-4ae5-88f2-f642527563bf\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.810428 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.810345    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"
Jul 29 21:22:03.954214 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.954143    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw"]
Jul 29 21:22:03.954465 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:22:03.954437    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb84e8424_a477_4ae5_88f2_f642527563bf.slice/crio-e197bb1e7606d95cec67586ada572509d8b211899cd68a31d04c244469bd8c3c WatchSource:0}: Error finding container e197bb1e7606d95cec67586ada572509d8b211899cd68a31d04c244469bd8c3c: Status 404 returned error can't find the container with id e197bb1e7606d95cec67586ada572509d8b211899cd68a31d04c244469bd8c3c
Jul 29 21:22:03.957099 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:03.957084    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:22:04.786361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:04.786326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw" event={"ID":"b84e8424-a477-4ae5-88f2-f642527563bf","Type":"ContainerStarted","Data":"e197bb1e7606d95cec67586ada572509d8b211899cd68a31d04c244469bd8c3c"}
Jul 29 21:22:05.219207 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:05.219130    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:05.319604 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:05.319565    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:07.804049 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.804009    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw" event={"ID":"b84e8424-a477-4ae5-88f2-f642527563bf","Type":"ContainerStarted","Data":"c7e6fb281b6be818cf8554ba8fb2dcb02ad1f2d5a4f3d7fa2e5474590d90a93d"}
Jul 29 21:22:07.834204 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.834134    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-ghflw" podStartSLOduration=1.922987432 podStartE2EDuration="4.834118364s" totalImagesPullingTime="2.911130932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:22:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:22:07.833020886 +0000 UTC m=+339.049798345" watchObservedRunningTime="2026-07-29 21:22:07.834118364 +0000 UTC m=+339.050895826"
Jul 29 21:22:07.909386 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.909352    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mgbxf"]
Jul 29 21:22:07.912640 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.912619    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:07.915379 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.915352    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7wz2j\""
Jul 29 21:22:07.915518 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.915353    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 21:22:07.915518 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.915362    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 21:22:07.922628 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.922600    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mgbxf"]
Jul 29 21:22:07.991950 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.991921    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5vg6\" (UniqueName: \"kubernetes.io/projected/4c92aa08-b5e9-45e7-8c88-673aefe32c16-kube-api-access-b5vg6\") pod \"cert-manager-webhook-6486ffd997-mgbxf\" (UID: \"4c92aa08-b5e9-45e7-8c88-673aefe32c16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:07.992119 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:07.991959    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c92aa08-b5e9-45e7-8c88-673aefe32c16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mgbxf\" (UID: \"4c92aa08-b5e9-45e7-8c88-673aefe32c16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:08.092958 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.092871    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5vg6\" (UniqueName: \"kubernetes.io/projected/4c92aa08-b5e9-45e7-8c88-673aefe32c16-kube-api-access-b5vg6\") pod \"cert-manager-webhook-6486ffd997-mgbxf\" (UID: \"4c92aa08-b5e9-45e7-8c88-673aefe32c16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:08.092958 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.092916    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c92aa08-b5e9-45e7-8c88-673aefe32c16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mgbxf\" (UID: \"4c92aa08-b5e9-45e7-8c88-673aefe32c16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:08.106972 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.106937    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c92aa08-b5e9-45e7-8c88-673aefe32c16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mgbxf\" (UID: \"4c92aa08-b5e9-45e7-8c88-673aefe32c16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:08.107120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.107079    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5vg6\" (UniqueName: \"kubernetes.io/projected/4c92aa08-b5e9-45e7-8c88-673aefe32c16-kube-api-access-b5vg6\") pod \"cert-manager-webhook-6486ffd997-mgbxf\" (UID: \"4c92aa08-b5e9-45e7-8c88-673aefe32c16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:08.222337 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.222293    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:08.387730 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.387649    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mgbxf"]
Jul 29 21:22:08.391332 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:22:08.391299    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c92aa08_b5e9_45e7_8c88_673aefe32c16.slice/crio-a0dd36c378fd51080ea3e54d7e925bc2c15472eee94ceb0795229691221a63fe WatchSource:0}: Error finding container a0dd36c378fd51080ea3e54d7e925bc2c15472eee94ceb0795229691221a63fe: Status 404 returned error can't find the container with id a0dd36c378fd51080ea3e54d7e925bc2c15472eee94ceb0795229691221a63fe
Jul 29 21:22:08.421769 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.421732    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:08.566483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.566445    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:08.810310 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:08.810275    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf" event={"ID":"4c92aa08-b5e9-45e7-8c88-673aefe32c16","Type":"ContainerStarted","Data":"a0dd36c378fd51080ea3e54d7e925bc2c15472eee94ceb0795229691221a63fe"}
Jul 29 21:22:09.579237 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:09.579196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:10.136461 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:10.136410    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:11.827375 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:11.827339    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf" event={"ID":"4c92aa08-b5e9-45e7-8c88-673aefe32c16","Type":"ContainerStarted","Data":"0be75b61b552d2f2262f8ba0f692010106443edf029108fb49cb477892d37675"}
Jul 29 21:22:11.827780 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:11.827404    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:11.844405 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:11.844354    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf" podStartSLOduration=2.09146604 podStartE2EDuration="4.844338753s" totalImagesPullingTime="2.752872713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:22:11.843624822 +0000 UTC m=+343.060402283" watchObservedRunningTime="2026-07-29 21:22:11.844338753 +0000 UTC m=+343.061116216"
Jul 29 21:22:11.952573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:11.952537    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:12.052772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:12.052731    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:14.989877 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:14.989839    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:15.051914 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:15.051877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:17.834052 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:17.834019    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mgbxf"
Jul 29 21:22:17.839153 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:17.839128    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:17.964372 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:17.964343    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:18.924768 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:18.924729    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:24.263836 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:24.263752    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:24.358840 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:24.358799    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:26.456671 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:26.456629    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:28.292479 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.292444    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k9p7d"]
Jul 29 21:22:28.296039 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.296021    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.299128 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.299108    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-d6pkg\""
Jul 29 21:22:28.308902 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.308874    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k9p7d"]
Jul 29 21:22:28.473601 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.473560    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-252hf\" (UniqueName: \"kubernetes.io/projected/b7ff25c5-bbde-4351-bda8-cf1dbed87cf2-kube-api-access-252hf\") pod \"cert-manager-55b4d954d5-k9p7d\" (UID: \"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2\") " pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.473788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.473648    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7ff25c5-bbde-4351-bda8-cf1dbed87cf2-bound-sa-token\") pod \"cert-manager-55b4d954d5-k9p7d\" (UID: \"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2\") " pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.574954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.574866    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7ff25c5-bbde-4351-bda8-cf1dbed87cf2-bound-sa-token\") pod \"cert-manager-55b4d954d5-k9p7d\" (UID: \"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2\") " pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.574954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.574920    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-252hf\" (UniqueName: \"kubernetes.io/projected/b7ff25c5-bbde-4351-bda8-cf1dbed87cf2-kube-api-access-252hf\") pod \"cert-manager-55b4d954d5-k9p7d\" (UID: \"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2\") " pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.587877 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.587837    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b7ff25c5-bbde-4351-bda8-cf1dbed87cf2-bound-sa-token\") pod \"cert-manager-55b4d954d5-k9p7d\" (UID: \"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2\") " pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.588007 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.587903    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-252hf\" (UniqueName: \"kubernetes.io/projected/b7ff25c5-bbde-4351-bda8-cf1dbed87cf2-kube-api-access-252hf\") pod \"cert-manager-55b4d954d5-k9p7d\" (UID: \"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2\") " pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.604808 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.604775    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k9p7d"
Jul 29 21:22:28.742019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.741988    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k9p7d"]
Jul 29 21:22:28.744028 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:22:28.744001    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7ff25c5_bbde_4351_bda8_cf1dbed87cf2.slice/crio-886101fd3295b9b29bedddd58df81bd59fcbc4a292c254d76eef40bad879aefb WatchSource:0}: Error finding container 886101fd3295b9b29bedddd58df81bd59fcbc4a292c254d76eef40bad879aefb: Status 404 returned error can't find the container with id 886101fd3295b9b29bedddd58df81bd59fcbc4a292c254d76eef40bad879aefb
Jul 29 21:22:28.905161 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.905068    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k9p7d" event={"ID":"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2","Type":"ContainerStarted","Data":"79101ffb22e778fcbdf964fc3744507f38b272e5c1680530a9af84f697af6153"}
Jul 29 21:22:28.905161 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.905113    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k9p7d" event={"ID":"b7ff25c5-bbde-4351-bda8-cf1dbed87cf2","Type":"ContainerStarted","Data":"886101fd3295b9b29bedddd58df81bd59fcbc4a292c254d76eef40bad879aefb"}
Jul 29 21:22:28.921583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:28.921533    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-k9p7d" podStartSLOduration=0.921515703 podStartE2EDuration="921.515703ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:22:28.921071556 +0000 UTC m=+360.137849020" watchObservedRunningTime="2026-07-29 21:22:28.921515703 +0000 UTC m=+360.138293164"
Jul 29 21:22:42.217409 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:42.217347    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:42.313894 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:42.313855    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:42.816352 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:42.816300    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.003188 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.003138    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.108282 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.108163    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.148929 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.148896    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"]
Jul 29 21:22:43.151372 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.151353    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.154403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.154365    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 21:22:43.154403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.154378    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 21:22:43.154635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.154368    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 21:22:43.154635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.154369    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sk4g6\""
Jul 29 21:22:43.154635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.154469    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 21:22:43.168262 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.168237    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"]
Jul 29 21:22:43.198555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.198521    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-428z7\" (UniqueName: \"kubernetes.io/projected/8c7f984e-894a-4f5b-9419-b16627818310-kube-api-access-428z7\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.198555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.198562    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c7f984e-894a-4f5b-9419-b16627818310-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.198781 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.198592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c7f984e-894a-4f5b-9419-b16627818310-manifests\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.198781 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.198660    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7f984e-894a-4f5b-9419-b16627818310-tmp\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.198781 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.198692    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c7f984e-894a-4f5b-9419-b16627818310-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.270063 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.270024    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.299514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.299476    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7f984e-894a-4f5b-9419-b16627818310-tmp\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.299688 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.299532    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c7f984e-894a-4f5b-9419-b16627818310-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.299688 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.299602    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-428z7\" (UniqueName: \"kubernetes.io/projected/8c7f984e-894a-4f5b-9419-b16627818310-kube-api-access-428z7\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.299688 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.299638    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c7f984e-894a-4f5b-9419-b16627818310-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.299857 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.299687    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c7f984e-894a-4f5b-9419-b16627818310-manifests\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.300081 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.300046    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8c7f984e-894a-4f5b-9419-b16627818310-manifests\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.301986 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.301962    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c7f984e-894a-4f5b-9419-b16627818310-tmp\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.302241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.302220    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c7f984e-894a-4f5b-9419-b16627818310-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.302318 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.302270    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8c7f984e-894a-4f5b-9419-b16627818310-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.312882 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.312845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-428z7\" (UniqueName: \"kubernetes.io/projected/8c7f984e-894a-4f5b-9419-b16627818310-kube-api-access-428z7\") pod \"opendatahub-operator-controller-manager-6b55745745-gwjsn\" (UID: \"8c7f984e-894a-4f5b-9419-b16627818310\") " pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.386459 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.386360    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.462367 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.462327    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:43.487522 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.487486    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.590232 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.590201    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.599633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.599608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"]
Jul 29 21:22:43.601410 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:22:43.601382    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7f984e_894a_4f5b_9419_b16627818310.slice/crio-ae0a32b62d81cbb38a2e0e2745f9e0e93268eb4eb1373399f93868017aa4c9e6 WatchSource:0}: Error finding container ae0a32b62d81cbb38a2e0e2745f9e0e93268eb4eb1373399f93868017aa4c9e6: Status 404 returned error can't find the container with id ae0a32b62d81cbb38a2e0e2745f9e0e93268eb4eb1373399f93868017aa4c9e6
Jul 29 21:22:43.687587 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.687554    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.791694 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.791654    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.898406 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.898371    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:43.972403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:43.972319    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn" event={"ID":"8c7f984e-894a-4f5b-9419-b16627818310","Type":"ContainerStarted","Data":"ae0a32b62d81cbb38a2e0e2745f9e0e93268eb4eb1373399f93868017aa4c9e6"}
Jul 29 21:22:46.989300 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:46.989218    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn" event={"ID":"8c7f984e-894a-4f5b-9419-b16627818310","Type":"ContainerStarted","Data":"6f288e03c0ead9c5657cae18bc9fbeda0531d93de3e4d4a7016b970d15abd486"}
Jul 29 21:22:47.995005 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:47.994969    2572 generic.go:356] "Generic (PLEG): container finished" podID="8c7f984e-894a-4f5b-9419-b16627818310" containerID="6f288e03c0ead9c5657cae18bc9fbeda0531d93de3e4d4a7016b970d15abd486" exitCode=0
Jul 29 21:22:47.995423 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:47.995059    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn" event={"ID":"8c7f984e-894a-4f5b-9419-b16627818310","Type":"ContainerDied","Data":"6f288e03c0ead9c5657cae18bc9fbeda0531d93de3e4d4a7016b970d15abd486"}
Jul 29 21:22:49.003827 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:49.003785    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn" event={"ID":"8c7f984e-894a-4f5b-9419-b16627818310","Type":"ContainerStarted","Data":"0dda53b7b7a29a5dbe3afbb6c9ec398e07f8406c0941338f2d294c6d03751467"}
Jul 29 21:22:49.004289 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:49.003929    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:22:49.028482 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:49.028407    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn" podStartSLOduration=1.592878698 podStartE2EDuration="6.028385374s" totalImagesPullingTime="3.405701685s" totalInitContainerRuntime="1.029804991s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:22:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:22:49.026236582 +0000 UTC m=+380.243014041" watchObservedRunningTime="2026-07-29 21:22:49.028385374 +0000 UTC m=+380.245162837"
Jul 29 21:22:52.659546 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:52.659491    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:52.759849 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:52.759808    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:55.202843 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:55.202802    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:22:55.329592 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:22:55.329550    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:00.009998 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:00.009960    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b55745745-gwjsn"
Jul 29 21:23:02.573631 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:02.573577    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:02.769798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:02.769758    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:03.215228 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.215190    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pffpc"]
Jul 29 21:23:03.222459 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.222433    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.225364 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.225342    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 21:23:03.225747 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.225725    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vctnq\""
Jul 29 21:23:03.245406 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.245376    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pffpc"]
Jul 29 21:23:03.314485 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.314441    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:03.374329 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.374289    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58cb8084-7ecc-40ab-a467-ca14d2e67719-cert\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.374540 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.374341    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xszr9\" (UniqueName: \"kubernetes.io/projected/58cb8084-7ecc-40ab-a467-ca14d2e67719-kube-api-access-xszr9\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.438317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.438254    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:03.475425 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.475322    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58cb8084-7ecc-40ab-a467-ca14d2e67719-cert\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.475425 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.475375    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xszr9\" (UniqueName: \"kubernetes.io/projected/58cb8084-7ecc-40ab-a467-ca14d2e67719-kube-api-access-xszr9\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.475676 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:23:03.475483    2572 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 29 21:23:03.475676 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:23:03.475559    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58cb8084-7ecc-40ab-a467-ca14d2e67719-cert podName:58cb8084-7ecc-40ab-a467-ca14d2e67719 nodeName:}" failed. No retries permitted until 2026-07-29 21:23:03.975541845 +0000 UTC m=+395.192319285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/58cb8084-7ecc-40ab-a467-ca14d2e67719-cert") pod "odh-model-controller-7998c67dc4-pffpc" (UID: "58cb8084-7ecc-40ab-a467-ca14d2e67719") : secret "odh-model-controller-webhook-cert" not found
Jul 29 21:23:03.485938 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.485887    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xszr9\" (UniqueName: \"kubernetes.io/projected/58cb8084-7ecc-40ab-a467-ca14d2e67719-kube-api-access-xszr9\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.563427 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.563392    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:03.949297 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.949267    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mpcf4"]
Jul 29 21:23:03.951622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.951604    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:03.953710 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.953687    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 29 21:23:03.953850 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.953785    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-qxznc\""
Jul 29 21:23:03.962461 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.962433    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mpcf4"]
Jul 29 21:23:03.967743 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.967708    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:03.980239 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.980206    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58cb8084-7ecc-40ab-a467-ca14d2e67719-cert\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:03.983540 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:03.983507    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/58cb8084-7ecc-40ab-a467-ca14d2e67719-cert\") pod \"odh-model-controller-7998c67dc4-pffpc\" (UID: \"58cb8084-7ecc-40ab-a467-ca14d2e67719\") " pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:04.080682 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.080639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f5c933-cd6a-40ef-9a57-5d378f513282-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.080863 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.080686    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpqv\" (UniqueName: \"kubernetes.io/projected/38f5c933-cd6a-40ef-9a57-5d378f513282-kube-api-access-kmpqv\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.133701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.133667    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:04.181662 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.181609    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f5c933-cd6a-40ef-9a57-5d378f513282-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.181861 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.181682    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpqv\" (UniqueName: \"kubernetes.io/projected/38f5c933-cd6a-40ef-9a57-5d378f513282-kube-api-access-kmpqv\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.181861 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:23:04.181762    2572 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 29 21:23:04.181861 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:23:04.181832    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38f5c933-cd6a-40ef-9a57-5d378f513282-tls-certs podName:38f5c933-cd6a-40ef-9a57-5d378f513282 nodeName:}" failed. No retries permitted until 2026-07-29 21:23:04.681814692 +0000 UTC m=+395.898592132 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/38f5c933-cd6a-40ef-9a57-5d378f513282-tls-certs") pod "model-serving-api-54d89bd5fc-mpcf4" (UID: "38f5c933-cd6a-40ef-9a57-5d378f513282") : secret "model-serving-api-tls" not found
Jul 29 21:23:04.201305 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.201162    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpqv\" (UniqueName: \"kubernetes.io/projected/38f5c933-cd6a-40ef-9a57-5d378f513282-kube-api-access-kmpqv\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.298484 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.297475    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pffpc"]
Jul 29 21:23:04.300373 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:23:04.300342    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58cb8084_7ecc_40ab_a467_ca14d2e67719.slice/crio-8ee543fa1161f1668bd0bd4f2bbd64f2a13b41b7df5b7db6a33d4d0c6df3f877 WatchSource:0}: Error finding container 8ee543fa1161f1668bd0bd4f2bbd64f2a13b41b7df5b7db6a33d4d0c6df3f877: Status 404 returned error can't find the container with id 8ee543fa1161f1668bd0bd4f2bbd64f2a13b41b7df5b7db6a33d4d0c6df3f877
Jul 29 21:23:04.350529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.350493    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:04.686251 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.686203    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f5c933-cd6a-40ef-9a57-5d378f513282-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.688734 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.688692    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f5c933-cd6a-40ef-9a57-5d378f513282-tls-certs\") pod \"model-serving-api-54d89bd5fc-mpcf4\" (UID: \"38f5c933-cd6a-40ef-9a57-5d378f513282\") " pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.864130 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.864096    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:04.998934 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:04.998907    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-mpcf4"]
Jul 29 21:23:05.001381 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:23:05.001339    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38f5c933_cd6a_40ef_9a57_5d378f513282.slice/crio-31c67a38152b88f87dab07f2b4ef1e749abb51a0ff2ad00687a9167cad4f2cb6 WatchSource:0}: Error finding container 31c67a38152b88f87dab07f2b4ef1e749abb51a0ff2ad00687a9167cad4f2cb6: Status 404 returned error can't find the container with id 31c67a38152b88f87dab07f2b4ef1e749abb51a0ff2ad00687a9167cad4f2cb6
Jul 29 21:23:05.079907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:05.079867    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc" event={"ID":"58cb8084-7ecc-40ab-a467-ca14d2e67719","Type":"ContainerStarted","Data":"8ee543fa1161f1668bd0bd4f2bbd64f2a13b41b7df5b7db6a33d4d0c6df3f877"}
Jul 29 21:23:05.081311 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:05.081276    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4" event={"ID":"38f5c933-cd6a-40ef-9a57-5d378f513282","Type":"ContainerStarted","Data":"31c67a38152b88f87dab07f2b4ef1e749abb51a0ff2ad00687a9167cad4f2cb6"}
Jul 29 21:23:08.102386 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.102348    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc" event={"ID":"58cb8084-7ecc-40ab-a467-ca14d2e67719","Type":"ContainerStarted","Data":"158078f9c62b45fcb0aba157646eb664bfae91ff82c8ea6ada0d1d0b15fcb2bf"}
Jul 29 21:23:08.102845 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.102532    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:08.104009 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.103986    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4" event={"ID":"38f5c933-cd6a-40ef-9a57-5d378f513282","Type":"ContainerStarted","Data":"5677c171637ac8f04c335224d6ec597cf69979fed7f64be1f9b707aff5bacd2f"}
Jul 29 21:23:08.104137 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.104106    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:08.120669 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.120622    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc" podStartSLOduration=1.4922848659999999 podStartE2EDuration="5.120608923s" totalImagesPullingTime="3.628324057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:23:08.118907223 +0000 UTC m=+399.335684698" watchObservedRunningTime="2026-07-29 21:23:08.120608923 +0000 UTC m=+399.337386386"
Jul 29 21:23:08.135463 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.135378    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4" podStartSLOduration=2.255425748 podStartE2EDuration="5.135365522s" totalImagesPullingTime="2.879939774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:23:08.134152209 +0000 UTC m=+399.350929670" watchObservedRunningTime="2026-07-29 21:23:08.135365522 +0000 UTC m=+399.352143031"
Jul 29 21:23:08.560037 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.559998    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:08.622529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.622492    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"]
Jul 29 21:23:08.625960 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.625940    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.628549 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.628503    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 21:23:08.628706 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.628555    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 21:23:08.628706 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.628599    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 21:23:08.628842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.628825    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 21:23:08.629327 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.629312    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xxzjf\""
Jul 29 21:23:08.637260 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.637229    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"]
Jul 29 21:23:08.637396 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.637371    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:08.722560 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.722518    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88rfq\" (UniqueName: \"kubernetes.io/projected/dbb736d7-8d40-4c69-990b-a3db196259aa-kube-api-access-88rfq\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.722757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.722570    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbb736d7-8d40-4c69-990b-a3db196259aa-tmp\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.722757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.722607    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb736d7-8d40-4c69-990b-a3db196259aa-tls-certs\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.823760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.823672    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88rfq\" (UniqueName: \"kubernetes.io/projected/dbb736d7-8d40-4c69-990b-a3db196259aa-kube-api-access-88rfq\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.823760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.823723    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbb736d7-8d40-4c69-990b-a3db196259aa-tmp\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.823760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.823750    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb736d7-8d40-4c69-990b-a3db196259aa-tls-certs\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.826083 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.826053    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dbb736d7-8d40-4c69-990b-a3db196259aa-tmp\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.826330 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.826309    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb736d7-8d40-4c69-990b-a3db196259aa-tls-certs\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.832684 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.832656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88rfq\" (UniqueName: \"kubernetes.io/projected/dbb736d7-8d40-4c69-990b-a3db196259aa-kube-api-access-88rfq\") pod \"kube-auth-proxy-787cd99557-fzfmn\" (UID: \"dbb736d7-8d40-4c69-990b-a3db196259aa\") " pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:08.936441 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:08.936408    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"
Jul 29 21:23:09.083113 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:09.083075    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-787cd99557-fzfmn"]
Jul 29 21:23:09.084218 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:23:09.084185    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb736d7_8d40_4c69_990b_a3db196259aa.slice/crio-1ff197294f4776df89f2c04dd49d912d03a004664f0cdda580e400696148dbb3 WatchSource:0}: Error finding container 1ff197294f4776df89f2c04dd49d912d03a004664f0cdda580e400696148dbb3: Status 404 returned error can't find the container with id 1ff197294f4776df89f2c04dd49d912d03a004664f0cdda580e400696148dbb3
Jul 29 21:23:09.110896 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:09.110857    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn" event={"ID":"dbb736d7-8d40-4c69-990b-a3db196259aa","Type":"ContainerStarted","Data":"1ff197294f4776df89f2c04dd49d912d03a004664f0cdda580e400696148dbb3"}
Jul 29 21:23:10.042981 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.042936    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:10.043777 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.043750    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"]
Jul 29 21:23:10.047324 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.047295    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.049947 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.049911    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 21:23:10.050104 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.049955    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 21:23:10.050104 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.049912    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vkt9s\""
Jul 29 21:23:10.050318 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.050255    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 21:23:10.050318 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.050277    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 21:23:10.068882 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.068849    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"]
Jul 29 21:23:10.120550 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.120515    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:10.135125 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135094    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.135312 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135133    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f28757-ac1c-473c-8b20-5680805dbaf5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.135312 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135241    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqb4q\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-kube-api-access-bqb4q\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.135423 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135307    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.135423 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135337    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.135518 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135437    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.135518 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.135491    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236026 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.235989    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236026 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236030    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236247 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236056    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236247 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236084    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f28757-ac1c-473c-8b20-5680805dbaf5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236247 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236104    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqb4q\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-kube-api-access-bqb4q\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236247 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236136    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236247 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236160    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.236865 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.236840    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.238266 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.238243    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f28757-ac1c-473c-8b20-5680805dbaf5-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.238514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.238484    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.238657 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.238641    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.238694 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.238681    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.244139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.244114    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.244594 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.244577    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqb4q\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-kube-api-access-bqb4q\") pod \"istiod-openshift-gateway-7b7995d5cd-tgd2j\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.359138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.359059    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:10.533843 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:10.533813    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"]
Jul 29 21:23:10.536147 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:23:10.536116    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f28757_ac1c_473c_8b20_5680805dbaf5.slice/crio-7255c94e6fb0528eb269c8a31a705b1e02a733c4503b17a411cc1aa5f76c4e99 WatchSource:0}: Error finding container 7255c94e6fb0528eb269c8a31a705b1e02a733c4503b17a411cc1aa5f76c4e99: Status 404 returned error can't find the container with id 7255c94e6fb0528eb269c8a31a705b1e02a733c4503b17a411cc1aa5f76c4e99
Jul 29 21:23:11.123380 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:11.123338    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" event={"ID":"b1f28757-ac1c-473c-8b20-5680805dbaf5","Type":"ContainerStarted","Data":"7255c94e6fb0528eb269c8a31a705b1e02a733c4503b17a411cc1aa5f76c4e99"}
Jul 29 21:23:11.749874 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:11.749836    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:12.132972 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:12.132877    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn" event={"ID":"dbb736d7-8d40-4c69-990b-a3db196259aa","Type":"ContainerStarted","Data":"b1dc02f87ce087328b760916b51dc34685edc17a5a96f507adc81564c6130b05"}
Jul 29 21:23:12.184128 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:12.184059    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-787cd99557-fzfmn" podStartSLOduration=1.458572329 podStartE2EDuration="4.184036615s" totalImagesPullingTime="2.725464286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:23:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:23:12.181344246 +0000 UTC m=+403.398121710" watchObservedRunningTime="2026-07-29 21:23:12.184036615 +0000 UTC m=+403.400814078"
Jul 29 21:23:13.540121 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:13.540076    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:13.643840 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:13.643802    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 21:23:13.643954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:13.643872    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 21:23:14.145899 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.145866    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" event={"ID":"b1f28757-ac1c-473c-8b20-5680805dbaf5","Type":"ContainerStarted","Data":"6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b"}
Jul 29 21:23:14.146087 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.146015    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:14.147670 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.147634    2572 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-tgd2j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 21:23:14.147798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.147697    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 21:23:14.232187 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.232120    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" podStartSLOduration=1.127488901 podStartE2EDuration="4.232102332s" totalImagesPullingTime="3.104613431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:23:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:23:14.226650882 +0000 UTC m=+405.443428344" watchObservedRunningTime="2026-07-29 21:23:14.232102332 +0000 UTC m=+405.448879793"
Jul 29 21:23:14.244150 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.244118    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:14.616797 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.616755    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:14.753631 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.753585    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:14.895764 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:14.895657    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:15.060710 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:15.060674    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:15.150781 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:15.150691    2572 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-tgd2j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 21:23:15.150953 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:15.150853    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 21:23:15.508871 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:15.508837    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:16.000726 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:16.000686    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:16.098621 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:16.098574    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:16.310650 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:16.310557    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:16.627975 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:16.627870    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:16.890634 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:16.890536    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:23:18.150882 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:18.150848    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:19.113029 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:19.112991    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pffpc"
Jul 29 21:23:19.114870 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:19.114837    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-mpcf4"
Jul 29 21:23:21.707677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:21.707635    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"]
Jul 29 21:23:21.708161 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:21.707848    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerName="discovery" containerID="cri-o://6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b" gracePeriod=30
Jul 29 21:23:21.955851 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:21.955820    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:22.052194 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052098    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-ca-configmap\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052194 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052141    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqb4q\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-kube-api-access-bqb4q\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052194 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052157    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-cacerts\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052443 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052310    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-kubeconfig\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052443 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052371    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-dns-cert\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052540 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052502    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-token\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052606 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052553    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f28757-ac1c-473c-8b20-5680805dbaf5-local-certs\") pod \"b1f28757-ac1c-473c-8b20-5680805dbaf5\" (UID: \"b1f28757-ac1c-473c-8b20-5680805dbaf5\") "
Jul 29 21:23:22.052656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052599    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:23:22.052892 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.052858    2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-ca-configmap\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.054855 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.054812    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-kube-api-access-bqb4q" (OuterVolumeSpecName: "kube-api-access-bqb4q") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "kube-api-access-bqb4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:23:22.054966 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.054874    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:23:22.054966 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.054920    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-cacerts" (OuterVolumeSpecName: "cacerts") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:23:22.054966 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.054951    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:23:22.055144 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.055011    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:23:22.055144 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.055086    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1f28757-ac1c-473c-8b20-5680805dbaf5-local-certs" (OuterVolumeSpecName: "local-certs") pod "b1f28757-ac1c-473c-8b20-5680805dbaf5" (UID: "b1f28757-ac1c-473c-8b20-5680805dbaf5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:23:22.153849 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.153809    2572 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1f28757-ac1c-473c-8b20-5680805dbaf5-local-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.153849 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.153844    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqb4q\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-kube-api-access-bqb4q\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.153849 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.153859    2572 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-cacerts\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.154082 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.153872    2572 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-kubeconfig\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.154082 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.153885    2572 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-csr-dns-cert\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.154082 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.153898    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1f28757-ac1c-473c-8b20-5680805dbaf5-istio-token\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:23:22.185504 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.185471    2572 generic.go:356] "Generic (PLEG): container finished" podID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerID="6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b" exitCode=0
Jul 29 21:23:22.185659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.185521    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" event={"ID":"b1f28757-ac1c-473c-8b20-5680805dbaf5","Type":"ContainerDied","Data":"6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b"}
Jul 29 21:23:22.185659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.185531    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"
Jul 29 21:23:22.185659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.185587    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j" event={"ID":"b1f28757-ac1c-473c-8b20-5680805dbaf5","Type":"ContainerDied","Data":"7255c94e6fb0528eb269c8a31a705b1e02a733c4503b17a411cc1aa5f76c4e99"}
Jul 29 21:23:22.185659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.185607    2572 scope.go:117] "RemoveContainer" containerID="6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b"
Jul 29 21:23:22.194550 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.194528    2572 scope.go:117] "RemoveContainer" containerID="6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b"
Jul 29 21:23:22.194819 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:23:22.194792    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b\": container with ID starting with 6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b not found: ID does not exist" containerID="6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b"
Jul 29 21:23:22.194912 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.194828    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b"} err="failed to get container status \"6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b\": rpc error: code = NotFound desc = could not find container \"6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b\": container with ID starting with 6349239599a4ea656a4e80799e378b6011057d6408f1b5ef499f0840481e210b not found: ID does not exist"
Jul 29 21:23:22.210690 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.210662    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"]
Jul 29 21:23:22.212924 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:22.212904    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tgd2j"]
Jul 29 21:23:23.303444 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:23.303408    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" path="/var/lib/kubelet/pods/b1f28757-ac1c-473c-8b20-5680805dbaf5/volumes"
Jul 29 21:23:58.416491 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:23:58.416399    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:03.990012 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:03.989975    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:04.089897 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.089861    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:04.128207 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.128164    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tjc8n"]
Jul 29 21:24:04.128565 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.128548    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerName="discovery"
Jul 29 21:24:04.128622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.128570    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerName="discovery"
Jul 29 21:24:04.128672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.128662    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f28757-ac1c-473c-8b20-5680805dbaf5" containerName="discovery"
Jul 29 21:24:04.130958 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.130944    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:04.134453 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.134430    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 21:24:04.135134 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.135110    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 21:24:04.135504 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.135480    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-blzv4\""
Jul 29 21:24:04.143826 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.143802    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tjc8n"]
Jul 29 21:24:04.196821 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.196788    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:04.316104 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.316004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5h79\" (UniqueName: \"kubernetes.io/projected/e8100b00-15ac-42ab-9c75-fb7863067b01-kube-api-access-m5h79\") pod \"kuadrant-operator-catalog-tjc8n\" (UID: \"e8100b00-15ac-42ab-9c75-fb7863067b01\") " pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:04.417436 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.417396    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5h79\" (UniqueName: \"kubernetes.io/projected/e8100b00-15ac-42ab-9c75-fb7863067b01-kube-api-access-m5h79\") pod \"kuadrant-operator-catalog-tjc8n\" (UID: \"e8100b00-15ac-42ab-9c75-fb7863067b01\") " pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:04.432131 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.432090    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5h79\" (UniqueName: \"kubernetes.io/projected/e8100b00-15ac-42ab-9c75-fb7863067b01-kube-api-access-m5h79\") pod \"kuadrant-operator-catalog-tjc8n\" (UID: \"e8100b00-15ac-42ab-9c75-fb7863067b01\") " pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:04.442038 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.442009    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:04.479538 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.479497    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tjc8n"]
Jul 29 21:24:04.581789 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.581762    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tjc8n"]
Jul 29 21:24:04.584163 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:04.584134    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8100b00_15ac_42ab_9c75_fb7863067b01.slice/crio-cb406d6a8c27db97ae406882b6754bacbd72b608e63d2c6df25540d9b737a934 WatchSource:0}: Error finding container cb406d6a8c27db97ae406882b6754bacbd72b608e63d2c6df25540d9b737a934: Status 404 returned error can't find the container with id cb406d6a8c27db97ae406882b6754bacbd72b608e63d2c6df25540d9b737a934
Jul 29 21:24:04.690210 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.690158    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cgp8k"]
Jul 29 21:24:04.693337 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.693316    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:04.715812 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.715772    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cgp8k"]
Jul 29 21:24:04.821120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.821076    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgkkn\" (UniqueName: \"kubernetes.io/projected/db724bb6-3ffe-4cac-a364-569c44b0f199-kube-api-access-qgkkn\") pod \"kuadrant-operator-catalog-cgp8k\" (UID: \"db724bb6-3ffe-4cac-a364-569c44b0f199\") " pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:04.922580 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.922484    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgkkn\" (UniqueName: \"kubernetes.io/projected/db724bb6-3ffe-4cac-a364-569c44b0f199-kube-api-access-qgkkn\") pod \"kuadrant-operator-catalog-cgp8k\" (UID: \"db724bb6-3ffe-4cac-a364-569c44b0f199\") " pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:04.930758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.930727    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:04.934572 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:04.934545    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgkkn\" (UniqueName: \"kubernetes.io/projected/db724bb6-3ffe-4cac-a364-569c44b0f199-kube-api-access-qgkkn\") pod \"kuadrant-operator-catalog-cgp8k\" (UID: \"db724bb6-3ffe-4cac-a364-569c44b0f199\") " pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:05.003534 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:05.003490    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:05.140733 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:05.140680    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:05.145381 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:05.145357    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cgp8k"]
Jul 29 21:24:05.146690 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:05.146657    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb724bb6_3ffe_4cac_a364_569c44b0f199.slice/crio-c4c696ee649d3255f629a5eb2ea60553508c4d5f6408dee7aa7030d5c9f5d6d0 WatchSource:0}: Error finding container c4c696ee649d3255f629a5eb2ea60553508c4d5f6408dee7aa7030d5c9f5d6d0: Status 404 returned error can't find the container with id c4c696ee649d3255f629a5eb2ea60553508c4d5f6408dee7aa7030d5c9f5d6d0
Jul 29 21:24:05.381627 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:05.381586    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k" event={"ID":"db724bb6-3ffe-4cac-a364-569c44b0f199","Type":"ContainerStarted","Data":"c4c696ee649d3255f629a5eb2ea60553508c4d5f6408dee7aa7030d5c9f5d6d0"}
Jul 29 21:24:05.382957 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:05.382921    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n" event={"ID":"e8100b00-15ac-42ab-9c75-fb7863067b01","Type":"ContainerStarted","Data":"cb406d6a8c27db97ae406882b6754bacbd72b608e63d2c6df25540d9b737a934"}
Jul 29 21:24:07.394150 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.394101    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n" event={"ID":"e8100b00-15ac-42ab-9c75-fb7863067b01","Type":"ContainerStarted","Data":"66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f"}
Jul 29 21:24:07.394616 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.394144    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n" podUID="e8100b00-15ac-42ab-9c75-fb7863067b01" containerName="registry-server" containerID="cri-o://66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f" gracePeriod=2
Jul 29 21:24:07.395708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.395686    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k" event={"ID":"db724bb6-3ffe-4cac-a364-569c44b0f199","Type":"ContainerStarted","Data":"2f1685073531de9afd35746e91afc267631a97ab7f2f830a78765484c7ab86c3"}
Jul 29 21:24:07.411557 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.411513    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n" podStartSLOduration=1.3030894960000001 podStartE2EDuration="3.411497043s" totalImagesPullingTime="2.108407547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:24:07.410541518 +0000 UTC m=+458.627318983" watchObservedRunningTime="2026-07-29 21:24:07.411497043 +0000 UTC m=+458.628274505"
Jul 29 21:24:07.427012 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.426961    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k" podStartSLOduration=1.8776429970000001 podStartE2EDuration="3.426946144s" totalImagesPullingTime="1.549303147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:24:07.425228822 +0000 UTC m=+458.642006283" watchObservedRunningTime="2026-07-29 21:24:07.426946144 +0000 UTC m=+458.643723607"
Jul 29 21:24:07.648917 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.648818    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:07.748783 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.748751    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5h79\" (UniqueName: \"kubernetes.io/projected/e8100b00-15ac-42ab-9c75-fb7863067b01-kube-api-access-m5h79\") pod \"e8100b00-15ac-42ab-9c75-fb7863067b01\" (UID: \"e8100b00-15ac-42ab-9c75-fb7863067b01\") "
Jul 29 21:24:07.750906 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.750872    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8100b00-15ac-42ab-9c75-fb7863067b01-kube-api-access-m5h79" (OuterVolumeSpecName: "kube-api-access-m5h79") pod "e8100b00-15ac-42ab-9c75-fb7863067b01" (UID: "e8100b00-15ac-42ab-9c75-fb7863067b01"). InnerVolumeSpecName "kube-api-access-m5h79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:24:07.849813 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:07.849774    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5h79\" (UniqueName: \"kubernetes.io/projected/e8100b00-15ac-42ab-9c75-fb7863067b01-kube-api-access-m5h79\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:08.401410 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.401372    2572 generic.go:356] "Generic (PLEG): container finished" podID="e8100b00-15ac-42ab-9c75-fb7863067b01" containerID="66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f" exitCode=0
Jul 29 21:24:08.401410 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.401412    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n"
Jul 29 21:24:08.401961 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.401453    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n" event={"ID":"e8100b00-15ac-42ab-9c75-fb7863067b01","Type":"ContainerDied","Data":"66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f"}
Jul 29 21:24:08.401961 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.401505    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-tjc8n" event={"ID":"e8100b00-15ac-42ab-9c75-fb7863067b01","Type":"ContainerDied","Data":"cb406d6a8c27db97ae406882b6754bacbd72b608e63d2c6df25540d9b737a934"}
Jul 29 21:24:08.401961 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.401527    2572 scope.go:117] "RemoveContainer" containerID="66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f"
Jul 29 21:24:08.410444 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.410422    2572 scope.go:117] "RemoveContainer" containerID="66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f"
Jul 29 21:24:08.410711 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:24:08.410692    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f\": container with ID starting with 66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f not found: ID does not exist" containerID="66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f"
Jul 29 21:24:08.410758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.410721    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f"} err="failed to get container status \"66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f\": rpc error: code = NotFound desc = could not find container \"66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f\": container with ID starting with 66bcb64176e0267612555790dec79c7ec4b4ed95f3141d751805d4bd5731251f not found: ID does not exist"
Jul 29 21:24:08.422465 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.422431    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tjc8n"]
Jul 29 21:24:08.429248 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:08.429223    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-tjc8n"]
Jul 29 21:24:09.312774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:09.312732    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8100b00-15ac-42ab-9c75-fb7863067b01" path="/var/lib/kubelet/pods/e8100b00-15ac-42ab-9c75-fb7863067b01/volumes"
Jul 29 21:24:15.003913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:15.003873    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:15.004377 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:15.003941    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:15.026144 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:15.026106    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:15.454346 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:15.454318    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-cgp8k"
Jul 29 21:24:20.142203 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.142152    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"]
Jul 29 21:24:20.142595 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.142501    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8100b00-15ac-42ab-9c75-fb7863067b01" containerName="registry-server"
Jul 29 21:24:20.142595 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.142512    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8100b00-15ac-42ab-9c75-fb7863067b01" containerName="registry-server"
Jul 29 21:24:20.142595 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.142572    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8100b00-15ac-42ab-9c75-fb7863067b01" containerName="registry-server"
Jul 29 21:24:20.147238 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.147217    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.149946 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.149921    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ztd8p\""
Jul 29 21:24:20.156327 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.156303    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"]
Jul 29 21:24:20.231605 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.231566    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8hxx\" (UniqueName: \"kubernetes.io/projected/77c2f682-844f-4e0c-83be-f66b9deb82da-kube-api-access-n8hxx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.231768 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.231626    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.231768 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.231692    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.332161 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.332113    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8hxx\" (UniqueName: \"kubernetes.io/projected/77c2f682-844f-4e0c-83be-f66b9deb82da-kube-api-access-n8hxx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.332397 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.332248    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.332397 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.332295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.332676 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.332657    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.332719 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.332696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.341429 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.341399    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8hxx\" (UniqueName: \"kubernetes.io/projected/77c2f682-844f-4e0c-83be-f66b9deb82da-kube-api-access-n8hxx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.455989 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.455961    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:20.592552 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.592526    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"]
Jul 29 21:24:20.595577 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:20.595541    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c2f682_844f_4e0c_83be_f66b9deb82da.slice/crio-8bc0ddf2cd43d67301905248f1e4504905cf955ab1e6429cfe1aabe2f1f298f4 WatchSource:0}: Error finding container 8bc0ddf2cd43d67301905248f1e4504905cf955ab1e6429cfe1aabe2f1f298f4: Status 404 returned error can't find the container with id 8bc0ddf2cd43d67301905248f1e4504905cf955ab1e6429cfe1aabe2f1f298f4
Jul 29 21:24:20.747934 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.747858    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"]
Jul 29 21:24:20.752487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.752471    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.761542 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.761514    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"]
Jul 29 21:24:20.836241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.836197    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.836417 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.836253    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.836417 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.836297    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bvv5\" (UniqueName: \"kubernetes.io/projected/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-kube-api-access-9bvv5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.937558 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.937514    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.937558 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.937561    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.937824 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.937598    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bvv5\" (UniqueName: \"kubernetes.io/projected/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-kube-api-access-9bvv5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.938009 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.937984    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.938090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.938072    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:20.946864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:20.946831    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bvv5\" (UniqueName: \"kubernetes.io/projected/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-kube-api-access-9bvv5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:21.062536 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.062429    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:21.124272 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.124238    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"]
Jul 29 21:24:21.129590 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.129567    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.136565 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.136530    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"]
Jul 29 21:24:21.139791 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.139754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.139919 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.139804    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9gbh\" (UniqueName: \"kubernetes.io/projected/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-kube-api-access-q9gbh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.139919 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.139872    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.205103 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.205076    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"]
Jul 29 21:24:21.207193 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:21.207149    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb15346_9fdf_4bf4_80dd_5b0f08dac116.slice/crio-2af2afd37db7edfd0328257bd0fe5a36dba415b39d722137f1ad81df0cbe6312 WatchSource:0}: Error finding container 2af2afd37db7edfd0328257bd0fe5a36dba415b39d722137f1ad81df0cbe6312: Status 404 returned error can't find the container with id 2af2afd37db7edfd0328257bd0fe5a36dba415b39d722137f1ad81df0cbe6312
Jul 29 21:24:21.241216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.241165    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.241347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.241264    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.241347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.241304    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9gbh\" (UniqueName: \"kubernetes.io/projected/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-kube-api-access-q9gbh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.241614 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.241587    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.241743 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.241656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.250138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.250109    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9gbh\" (UniqueName: \"kubernetes.io/projected/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-kube-api-access-q9gbh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.440442 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.440403    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:21.464291 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.464249    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g" event={"ID":"8cb15346-9fdf-4bf4-80dd-5b0f08dac116","Type":"ContainerStarted","Data":"2af2afd37db7edfd0328257bd0fe5a36dba415b39d722137f1ad81df0cbe6312"}
Jul 29 21:24:21.466943 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.466902    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x" event={"ID":"77c2f682-844f-4e0c-83be-f66b9deb82da","Type":"ContainerStarted","Data":"8bc0ddf2cd43d67301905248f1e4504905cf955ab1e6429cfe1aabe2f1f298f4"}
Jul 29 21:24:21.534206 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.534154    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"]
Jul 29 21:24:21.538940 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.538913    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.547096 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.545098    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfzpw\" (UniqueName: \"kubernetes.io/projected/fad08190-5525-494d-917b-cb5a39c4b510-kube-api-access-wfzpw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.547096 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.546074    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.547096 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.546114    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.548358 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.548334    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"]
Jul 29 21:24:21.602798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.602766    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"]
Jul 29 21:24:21.606181 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:21.606133    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7b3c7b_d5a6_454a_bd7c_8da8e65111bc.slice/crio-f1a49fd75e24679159de23f6bbe49100b74051e87ad15e80baac887e53a449af WatchSource:0}: Error finding container f1a49fd75e24679159de23f6bbe49100b74051e87ad15e80baac887e53a449af: Status 404 returned error can't find the container with id f1a49fd75e24679159de23f6bbe49100b74051e87ad15e80baac887e53a449af
Jul 29 21:24:21.647494 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.647460    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfzpw\" (UniqueName: \"kubernetes.io/projected/fad08190-5525-494d-917b-cb5a39c4b510-kube-api-access-wfzpw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.647656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.647589    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.647656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.647624    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.648040 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.648013    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.648141 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.648076    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.658113 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.658090    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfzpw\" (UniqueName: \"kubernetes.io/projected/fad08190-5525-494d-917b-cb5a39c4b510-kube-api-access-wfzpw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.855043 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.854962    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:21.997759 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:21.997716    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"]
Jul 29 21:24:21.999259 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:21.999226    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfad08190_5525_494d_917b_cb5a39c4b510.slice/crio-131f6f35f42968f2232760fc33aa6459c956efb76aacf06a8d035f8fa9479be0 WatchSource:0}: Error finding container 131f6f35f42968f2232760fc33aa6459c956efb76aacf06a8d035f8fa9479be0: Status 404 returned error can't find the container with id 131f6f35f42968f2232760fc33aa6459c956efb76aacf06a8d035f8fa9479be0
Jul 29 21:24:22.478268 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:22.478163    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd" event={"ID":"fad08190-5525-494d-917b-cb5a39c4b510","Type":"ContainerStarted","Data":"131f6f35f42968f2232760fc33aa6459c956efb76aacf06a8d035f8fa9479be0"}
Jul 29 21:24:22.480595 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:22.480537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77" event={"ID":"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc","Type":"ContainerStarted","Data":"f1a49fd75e24679159de23f6bbe49100b74051e87ad15e80baac887e53a449af"}
Jul 29 21:24:28.522030 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.521995    2572 generic.go:356] "Generic (PLEG): container finished" podID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerID="1b53def4e896a6ff7dee6b3c1202e49ce38c6368f86551bc8d706ed88458e54e" exitCode=0
Jul 29 21:24:28.522694 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.522083    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x" event={"ID":"77c2f682-844f-4e0c-83be-f66b9deb82da","Type":"ContainerDied","Data":"1b53def4e896a6ff7dee6b3c1202e49ce38c6368f86551bc8d706ed88458e54e"}
Jul 29 21:24:28.523899 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.523870    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerID="5166c7e511633e7e4881d4b1e18214625bf78af161c432fc8c7d22cfc9981cac" exitCode=0
Jul 29 21:24:28.523992 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.523947    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77" event={"ID":"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc","Type":"ContainerDied","Data":"5166c7e511633e7e4881d4b1e18214625bf78af161c432fc8c7d22cfc9981cac"}
Jul 29 21:24:28.525782 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.525757    2572 generic.go:356] "Generic (PLEG): container finished" podID="fad08190-5525-494d-917b-cb5a39c4b510" containerID="0414430f76a4b744e52956d035b16bc338d00028805e3c1e8c959ee6c7edce41" exitCode=0
Jul 29 21:24:28.525882 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.525838    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd" event={"ID":"fad08190-5525-494d-917b-cb5a39c4b510","Type":"ContainerDied","Data":"0414430f76a4b744e52956d035b16bc338d00028805e3c1e8c959ee6c7edce41"}
Jul 29 21:24:28.527966 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.527936    2572 generic.go:356] "Generic (PLEG): container finished" podID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerID="d7a84d158171255ccd46b031a8ef21711567b421c5d5e44c5f217b7117024e8a" exitCode=0
Jul 29 21:24:28.528061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:28.527978    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g" event={"ID":"8cb15346-9fdf-4bf4-80dd-5b0f08dac116","Type":"ContainerDied","Data":"d7a84d158171255ccd46b031a8ef21711567b421c5d5e44c5f217b7117024e8a"}
Jul 29 21:24:29.536067 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:29.536028    2572 generic.go:356] "Generic (PLEG): container finished" podID="fad08190-5525-494d-917b-cb5a39c4b510" containerID="2d265c8cf911ae041e84ac64789663217c73b5c4c486fab0f5f2464d746036ae" exitCode=0
Jul 29 21:24:29.536496 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:29.536100    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd" event={"ID":"fad08190-5525-494d-917b-cb5a39c4b510","Type":"ContainerDied","Data":"2d265c8cf911ae041e84ac64789663217c73b5c4c486fab0f5f2464d746036ae"}
Jul 29 21:24:30.544107 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:30.544013    2572 generic.go:356] "Generic (PLEG): container finished" podID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerID="951f285b5aa63122566c995268bde9c7659e001cb6371e1e98da55d8385126dc" exitCode=0
Jul 29 21:24:30.544561 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:30.544102    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g" event={"ID":"8cb15346-9fdf-4bf4-80dd-5b0f08dac116","Type":"ContainerDied","Data":"951f285b5aa63122566c995268bde9c7659e001cb6371e1e98da55d8385126dc"}
Jul 29 21:24:30.546343 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:30.546319    2572 generic.go:356] "Generic (PLEG): container finished" podID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerID="fc9c22610c5aaaeac1dd57aa23e334dc36a9a6003f3d3b827f2af7980db6bd7d" exitCode=0
Jul 29 21:24:30.546457 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:30.546418    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x" event={"ID":"77c2f682-844f-4e0c-83be-f66b9deb82da","Type":"ContainerDied","Data":"fc9c22610c5aaaeac1dd57aa23e334dc36a9a6003f3d3b827f2af7980db6bd7d"}
Jul 29 21:24:30.549047 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:30.549021    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerID="3c6522e9c02f5a8a899dbcebd43c36065c93e1bec4a2fbddc59c4cb7cbf02bea" exitCode=0
Jul 29 21:24:30.549151 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:30.549106    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77" event={"ID":"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc","Type":"ContainerDied","Data":"3c6522e9c02f5a8a899dbcebd43c36065c93e1bec4a2fbddc59c4cb7cbf02bea"}
Jul 29 21:24:34.818063 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:34.818023    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:38.610697 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.610658    2572 generic.go:356] "Generic (PLEG): container finished" podID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerID="8c9ba01feab9d40a0fb1613293721a40ec9352d11b8f2ecdc41f688ec46501b7" exitCode=0
Jul 29 21:24:38.611095 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.610737    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g" event={"ID":"8cb15346-9fdf-4bf4-80dd-5b0f08dac116","Type":"ContainerDied","Data":"8c9ba01feab9d40a0fb1613293721a40ec9352d11b8f2ecdc41f688ec46501b7"}
Jul 29 21:24:38.612807 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.612783    2572 generic.go:356] "Generic (PLEG): container finished" podID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerID="23a6dea99719c322754ea9cee04955dbdd2725d4bc42b305831bae2659b588ad" exitCode=0
Jul 29 21:24:38.612948 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.612863    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x" event={"ID":"77c2f682-844f-4e0c-83be-f66b9deb82da","Type":"ContainerDied","Data":"23a6dea99719c322754ea9cee04955dbdd2725d4bc42b305831bae2659b588ad"}
Jul 29 21:24:38.614874 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.614853    2572 generic.go:356] "Generic (PLEG): container finished" podID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerID="649c0e36cea99d85d4d113a42604a720f4beea6d14993fa9153b136e1b61e30e" exitCode=0
Jul 29 21:24:38.614979 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.614931    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77" event={"ID":"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc","Type":"ContainerDied","Data":"649c0e36cea99d85d4d113a42604a720f4beea6d14993fa9153b136e1b61e30e"}
Jul 29 21:24:38.616948 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.616927    2572 generic.go:356] "Generic (PLEG): container finished" podID="fad08190-5525-494d-917b-cb5a39c4b510" containerID="2f180e1953c334aa508b0094d1320ffcae1e1297a73b83abca929dae1086c42b" exitCode=0
Jul 29 21:24:38.617054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:38.617000    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd" event={"ID":"fad08190-5525-494d-917b-cb5a39c4b510","Type":"ContainerDied","Data":"2f180e1953c334aa508b0094d1320ffcae1e1297a73b83abca929dae1086c42b"}
Jul 29 21:24:39.768102 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.768065    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:39.812544 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.812520    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:39.816881 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.816861    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:39.826782 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.826760    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfzpw\" (UniqueName: \"kubernetes.io/projected/fad08190-5525-494d-917b-cb5a39c4b510-kube-api-access-wfzpw\") pod \"fad08190-5525-494d-917b-cb5a39c4b510\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") "
Jul 29 21:24:39.826922 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.826836    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-util\") pod \"fad08190-5525-494d-917b-cb5a39c4b510\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") "
Jul 29 21:24:39.826982 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.826943    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-bundle\") pod \"fad08190-5525-494d-917b-cb5a39c4b510\" (UID: \"fad08190-5525-494d-917b-cb5a39c4b510\") "
Jul 29 21:24:39.827374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.827349    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-bundle" (OuterVolumeSpecName: "bundle") pod "fad08190-5525-494d-917b-cb5a39c4b510" (UID: "fad08190-5525-494d-917b-cb5a39c4b510"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.828904 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.828879    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fad08190-5525-494d-917b-cb5a39c4b510-kube-api-access-wfzpw" (OuterVolumeSpecName: "kube-api-access-wfzpw") pod "fad08190-5525-494d-917b-cb5a39c4b510" (UID: "fad08190-5525-494d-917b-cb5a39c4b510"). InnerVolumeSpecName "kube-api-access-wfzpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:24:39.831673 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.831646    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-util" (OuterVolumeSpecName: "util") pod "fad08190-5525-494d-917b-cb5a39c4b510" (UID: "fad08190-5525-494d-917b-cb5a39c4b510"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.843111 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.843078    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:39.927790 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.927755    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-util\") pod \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") "
Jul 29 21:24:39.927963 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.927836    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-bundle\") pod \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") "
Jul 29 21:24:39.927963 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.927895    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-util\") pod \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") "
Jul 29 21:24:39.927963 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.927941    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-util\") pod \"77c2f682-844f-4e0c-83be-f66b9deb82da\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") "
Jul 29 21:24:39.928115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.927964    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-bundle\") pod \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") "
Jul 29 21:24:39.928115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.927993    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bvv5\" (UniqueName: \"kubernetes.io/projected/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-kube-api-access-9bvv5\") pod \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\" (UID: \"8cb15346-9fdf-4bf4-80dd-5b0f08dac116\") "
Jul 29 21:24:39.928115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928023    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8hxx\" (UniqueName: \"kubernetes.io/projected/77c2f682-844f-4e0c-83be-f66b9deb82da-kube-api-access-n8hxx\") pod \"77c2f682-844f-4e0c-83be-f66b9deb82da\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") "
Jul 29 21:24:39.928115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928056    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9gbh\" (UniqueName: \"kubernetes.io/projected/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-kube-api-access-q9gbh\") pod \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\" (UID: \"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc\") "
Jul 29 21:24:39.928115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928093    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-bundle\") pod \"77c2f682-844f-4e0c-83be-f66b9deb82da\" (UID: \"77c2f682-844f-4e0c-83be-f66b9deb82da\") "
Jul 29 21:24:39.928401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928380    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-bundle\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:39.928458 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928409    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfzpw\" (UniqueName: \"kubernetes.io/projected/fad08190-5525-494d-917b-cb5a39c4b510-kube-api-access-wfzpw\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:39.928458 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928423    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fad08190-5525-494d-917b-cb5a39c4b510-util\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:39.928560 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928502    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-bundle" (OuterVolumeSpecName: "bundle") pod "1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" (UID: "1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.928962 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928929    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-bundle" (OuterVolumeSpecName: "bundle") pod "8cb15346-9fdf-4bf4-80dd-5b0f08dac116" (UID: "8cb15346-9fdf-4bf4-80dd-5b0f08dac116"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.929059 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.928939    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-bundle" (OuterVolumeSpecName: "bundle") pod "77c2f682-844f-4e0c-83be-f66b9deb82da" (UID: "77c2f682-844f-4e0c-83be-f66b9deb82da"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.930999 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.930957    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-kube-api-access-9bvv5" (OuterVolumeSpecName: "kube-api-access-9bvv5") pod "8cb15346-9fdf-4bf4-80dd-5b0f08dac116" (UID: "8cb15346-9fdf-4bf4-80dd-5b0f08dac116"). InnerVolumeSpecName "kube-api-access-9bvv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:24:39.931123 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.931057    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-kube-api-access-q9gbh" (OuterVolumeSpecName: "kube-api-access-q9gbh") pod "1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" (UID: "1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc"). InnerVolumeSpecName "kube-api-access-q9gbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:24:39.931280 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.931251    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c2f682-844f-4e0c-83be-f66b9deb82da-kube-api-access-n8hxx" (OuterVolumeSpecName: "kube-api-access-n8hxx") pod "77c2f682-844f-4e0c-83be-f66b9deb82da" (UID: "77c2f682-844f-4e0c-83be-f66b9deb82da"). InnerVolumeSpecName "kube-api-access-n8hxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:24:39.933242 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.933213    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-util" (OuterVolumeSpecName: "util") pod "77c2f682-844f-4e0c-83be-f66b9deb82da" (UID: "77c2f682-844f-4e0c-83be-f66b9deb82da"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.933383 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.933360    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-util" (OuterVolumeSpecName: "util") pod "1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" (UID: "1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:39.934066 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:39.934046    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-util" (OuterVolumeSpecName: "util") pod "8cb15346-9fdf-4bf4-80dd-5b0f08dac116" (UID: "8cb15346-9fdf-4bf4-80dd-5b0f08dac116"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:24:40.029665 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029623    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-bundle\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029665 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029660    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bvv5\" (UniqueName: \"kubernetes.io/projected/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-kube-api-access-9bvv5\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029665 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029671    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8hxx\" (UniqueName: \"kubernetes.io/projected/77c2f682-844f-4e0c-83be-f66b9deb82da-kube-api-access-n8hxx\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029680    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9gbh\" (UniqueName: \"kubernetes.io/projected/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-kube-api-access-q9gbh\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029689    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-bundle\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029697    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-util\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029706    2572 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc-bundle\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029712    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8cb15346-9fdf-4bf4-80dd-5b0f08dac116-util\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.029907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.029719    2572 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/77c2f682-844f-4e0c-83be-f66b9deb82da-util\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:24:40.627928 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.627898    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd"
Jul 29 21:24:40.628108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.627934    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd" event={"ID":"fad08190-5525-494d-917b-cb5a39c4b510","Type":"ContainerDied","Data":"131f6f35f42968f2232760fc33aa6459c956efb76aacf06a8d035f8fa9479be0"}
Jul 29 21:24:40.628108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.627984    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="131f6f35f42968f2232760fc33aa6459c956efb76aacf06a8d035f8fa9479be0"
Jul 29 21:24:40.629841 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.629823    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g"
Jul 29 21:24:40.629841 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.629827    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g" event={"ID":"8cb15346-9fdf-4bf4-80dd-5b0f08dac116","Type":"ContainerDied","Data":"2af2afd37db7edfd0328257bd0fe5a36dba415b39d722137f1ad81df0cbe6312"}
Jul 29 21:24:40.630015 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.629867    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2af2afd37db7edfd0328257bd0fe5a36dba415b39d722137f1ad81df0cbe6312"
Jul 29 21:24:40.631751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.631733    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x"
Jul 29 21:24:40.631751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.631741    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x" event={"ID":"77c2f682-844f-4e0c-83be-f66b9deb82da","Type":"ContainerDied","Data":"8bc0ddf2cd43d67301905248f1e4504905cf955ab1e6429cfe1aabe2f1f298f4"}
Jul 29 21:24:40.631914 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.631777    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8bc0ddf2cd43d67301905248f1e4504905cf955ab1e6429cfe1aabe2f1f298f4"
Jul 29 21:24:40.633563 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.633543    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77" event={"ID":"1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc","Type":"ContainerDied","Data":"f1a49fd75e24679159de23f6bbe49100b74051e87ad15e80baac887e53a449af"}
Jul 29 21:24:40.633657 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.633574    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1a49fd75e24679159de23f6bbe49100b74051e87ad15e80baac887e53a449af"
Jul 29 21:24:40.633657 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:40.633583    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77"
Jul 29 21:24:44.215632 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:44.215584    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:45.540481 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:45.540432    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:45.638554 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:45.638518    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:45.698608 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:45.698576    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:46.426985 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.426954    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"]
Jul 29 21:24:46.427316 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427292    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="util"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427325    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="util"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427341    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="pull"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427349    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="pull"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427358    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="extract"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427365    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="extract"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427376    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="util"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427382    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="util"
Jul 29 21:24:46.427390 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427390    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="util"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427395    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="util"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427404    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427409    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427416    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="pull"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427420    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="pull"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427426    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427431    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427439    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="util"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427444    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="util"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427450    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="pull"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427455    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="pull"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427461    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="pull"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427465    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="pull"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427472    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427477    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427528    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="fad08190-5525-494d-917b-cb5a39c4b510" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427536    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427544    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cb15346-9fdf-4bf4-80dd-5b0f08dac116" containerName="extract"
Jul 29 21:24:46.427635 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.427550    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="77c2f682-844f-4e0c-83be-f66b9deb82da" containerName="extract"
Jul 29 21:24:46.430406 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.430388    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:24:46.434804 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.434786    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rb67q\""
Jul 29 21:24:46.446278 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.446254    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"]
Jul 29 21:24:46.586495 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.586452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn4sz\" (UniqueName: \"kubernetes.io/projected/b93759e7-3910-4067-a59c-988ab7106482-kube-api-access-dn4sz\") pod \"limitador-operator-controller-manager-859d7b78bd-22r72\" (UID: \"b93759e7-3910-4067-a59c-988ab7106482\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:24:46.687320 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.687290    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn4sz\" (UniqueName: \"kubernetes.io/projected/b93759e7-3910-4067-a59c-988ab7106482-kube-api-access-dn4sz\") pod \"limitador-operator-controller-manager-859d7b78bd-22r72\" (UID: \"b93759e7-3910-4067-a59c-988ab7106482\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:24:46.697985 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.697958    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn4sz\" (UniqueName: \"kubernetes.io/projected/b93759e7-3910-4067-a59c-988ab7106482-kube-api-access-dn4sz\") pod \"limitador-operator-controller-manager-859d7b78bd-22r72\" (UID: \"b93759e7-3910-4067-a59c-988ab7106482\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:24:46.739825 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.739794    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:24:46.869322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:46.869294    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"]
Jul 29 21:24:46.871412 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:46.871374    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93759e7_3910_4067_a59c_988ab7106482.slice/crio-139bc18e29f8df235fd1e56777885b2de267e21a3abd3a0392fc0ea0fd6a3d4a WatchSource:0}: Error finding container 139bc18e29f8df235fd1e56777885b2de267e21a3abd3a0392fc0ea0fd6a3d4a: Status 404 returned error can't find the container with id 139bc18e29f8df235fd1e56777885b2de267e21a3abd3a0392fc0ea0fd6a3d4a
Jul 29 21:24:47.213031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:47.212991    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:47.669140 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:47.669055    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72" event={"ID":"b93759e7-3910-4067-a59c-988ab7106482","Type":"ContainerStarted","Data":"139bc18e29f8df235fd1e56777885b2de267e21a3abd3a0392fc0ea0fd6a3d4a"}
Jul 29 21:24:48.020270 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:48.020217    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:49.682399 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:49.682354    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72" event={"ID":"b93759e7-3910-4067-a59c-988ab7106482","Type":"ContainerStarted","Data":"cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a"}
Jul 29 21:24:49.682807 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:49.682419    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:24:49.702757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:49.702700    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72" podStartSLOduration=1.954432009 podStartE2EDuration="3.702684786s" totalImagesPullingTime="1.748252777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:24:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:24:49.700884983 +0000 UTC m=+500.917662449" watchObservedRunningTime="2026-07-29 21:24:49.702684786 +0000 UTC m=+500.919462248"
Jul 29 21:24:50.673884 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.673843    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"]
Jul 29 21:24:50.676163 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.676147    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:24:50.678866 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.678843    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zgq55\""
Jul 29 21:24:50.691110 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.691084    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"]
Jul 29 21:24:50.820675 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.820641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k956n\" (UniqueName: \"kubernetes.io/projected/89fb5a08-2220-444e-a86a-7c57191ef16d-kube-api-access-k956n\") pod \"authorino-operator-75cc8d5cb7-7bvsf\" (UID: \"89fb5a08-2220-444e-a86a-7c57191ef16d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:24:50.921418 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.921384    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k956n\" (UniqueName: \"kubernetes.io/projected/89fb5a08-2220-444e-a86a-7c57191ef16d-kube-api-access-k956n\") pod \"authorino-operator-75cc8d5cb7-7bvsf\" (UID: \"89fb5a08-2220-444e-a86a-7c57191ef16d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:24:50.933403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.933375    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k956n\" (UniqueName: \"kubernetes.io/projected/89fb5a08-2220-444e-a86a-7c57191ef16d-kube-api-access-k956n\") pod \"authorino-operator-75cc8d5cb7-7bvsf\" (UID: \"89fb5a08-2220-444e-a86a-7c57191ef16d\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:24:50.986598 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:50.986565    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:24:51.155418 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:51.155391    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"]
Jul 29 21:24:51.157102 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:51.157073    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89fb5a08_2220_444e_a86a_7c57191ef16d.slice/crio-dbc153fb5124222f2ab2c55a33e44f1c89d236cd65b279d59f5cbd7458b7ce1c WatchSource:0}: Error finding container dbc153fb5124222f2ab2c55a33e44f1c89d236cd65b279d59f5cbd7458b7ce1c: Status 404 returned error can't find the container with id dbc153fb5124222f2ab2c55a33e44f1c89d236cd65b279d59f5cbd7458b7ce1c
Jul 29 21:24:51.659487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:51.659449    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:51.693070 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:51.693036    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf" event={"ID":"89fb5a08-2220-444e-a86a-7c57191ef16d","Type":"ContainerStarted","Data":"dbc153fb5124222f2ab2c55a33e44f1c89d236cd65b279d59f5cbd7458b7ce1c"}
Jul 29 21:24:51.843072 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:51.843036    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:53.445226 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.445156    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:53.855647 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.855616    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"]
Jul 29 21:24:53.859077 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.859044    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:24:53.861891 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.861867    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 29 21:24:53.862356 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.862342    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sqqmf\""
Jul 29 21:24:53.872745 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.872722    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"]
Jul 29 21:24:53.948108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:53.948070    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9jb\" (UniqueName: \"kubernetes.io/projected/4425f501-704a-4593-8dbc-52ef60091af9-kube-api-access-5j9jb\") pod \"dns-operator-controller-manager-864dd78c4c-df5tk\" (UID: \"4425f501-704a-4593-8dbc-52ef60091af9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:24:54.049296 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.049258    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9jb\" (UniqueName: \"kubernetes.io/projected/4425f501-704a-4593-8dbc-52ef60091af9-kube-api-access-5j9jb\") pod \"dns-operator-controller-manager-864dd78c4c-df5tk\" (UID: \"4425f501-704a-4593-8dbc-52ef60091af9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:24:54.064416 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.064351    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9jb\" (UniqueName: \"kubernetes.io/projected/4425f501-704a-4593-8dbc-52ef60091af9-kube-api-access-5j9jb\") pod \"dns-operator-controller-manager-864dd78c4c-df5tk\" (UID: \"4425f501-704a-4593-8dbc-52ef60091af9\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:24:54.168956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.168917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:24:54.305313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.305288    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"]
Jul 29 21:24:54.307029 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:54.306990    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4425f501_704a_4593_8dbc_52ef60091af9.slice/crio-3921a1257b3f8d6b52c7b656149ccdabac43ba7db047f8b52e22ddcfbebf38d0 WatchSource:0}: Error finding container 3921a1257b3f8d6b52c7b656149ccdabac43ba7db047f8b52e22ddcfbebf38d0: Status 404 returned error can't find the container with id 3921a1257b3f8d6b52c7b656149ccdabac43ba7db047f8b52e22ddcfbebf38d0
Jul 29 21:24:54.617888 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.617843    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:54.716622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.716581    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk" event={"ID":"4425f501-704a-4593-8dbc-52ef60091af9","Type":"ContainerStarted","Data":"3921a1257b3f8d6b52c7b656149ccdabac43ba7db047f8b52e22ddcfbebf38d0"}
Jul 29 21:24:54.718325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.718299    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf" event={"ID":"89fb5a08-2220-444e-a86a-7c57191ef16d","Type":"ContainerStarted","Data":"270bd845ac43d841b2841f61bd7ec03c4f78ed991d4ac5c285daebc5458746cc"}
Jul 29 21:24:54.718454 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.718431    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:24:54.742890 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.742838    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf" podStartSLOduration=2.096156884 podStartE2EDuration="4.742822045s" totalImagesPullingTime="2.646665161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:24:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:24:54.741810332 +0000 UTC m=+505.958587789" watchObservedRunningTime="2026-07-29 21:24:54.742822045 +0000 UTC m=+505.959599507"
Jul 29 21:24:54.914972 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:54.914861    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:56.649581 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:56.649547    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:56.731109 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:56.731076    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk" event={"ID":"4425f501-704a-4593-8dbc-52ef60091af9","Type":"ContainerStarted","Data":"62e7ad79f76af3ea3053ad0ca1771b6da032a2e406d58dc9202ee02fc49fee0f"}
Jul 29 21:24:56.731243 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:56.731219    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:24:56.754501 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:56.754435    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk" podStartSLOduration=1.394730618 podStartE2EDuration="3.754415285s" totalImagesPullingTime="2.359684667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:24:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:24:56.75306498 +0000 UTC m=+507.969842441" watchObservedRunningTime="2026-07-29 21:24:56.754415285 +0000 UTC m=+507.971192750"
Jul 29 21:24:58.775489 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.775451    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"]
Jul 29 21:24:58.778624 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.778602    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:58.780916 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.780892    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nm827\""
Jul 29 21:24:58.797367 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.797337    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"]
Jul 29 21:24:58.891772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.891729    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r9lr\" (UniqueName: \"kubernetes.io/projected/d514c30e-d844-4e08-9a90-a0f91329b77d-kube-api-access-6r9lr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:58.891954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.891809    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d514c30e-d844-4e08-9a90-a0f91329b77d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:58.992219 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.992180    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r9lr\" (UniqueName: \"kubernetes.io/projected/d514c30e-d844-4e08-9a90-a0f91329b77d-kube-api-access-6r9lr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:58.992395 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.992251    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d514c30e-d844-4e08-9a90-a0f91329b77d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:58.992677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:58.992656    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d514c30e-d844-4e08-9a90-a0f91329b77d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:59.000642 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:59.000608    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r9lr\" (UniqueName: \"kubernetes.io/projected/d514c30e-d844-4e08-9a90-a0f91329b77d-kube-api-access-6r9lr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:59.089816 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:59.089725    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:24:59.223567 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:59.223541    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"]
Jul 29 21:24:59.226511 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:24:59.226484    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd514c30e_d844_4e08_9a90_a0f91329b77d.slice/crio-f0a9db1ba2abcb2b2bb06643e2615d4d22f27daa316d5290e6cc99621c70913e WatchSource:0}: Error finding container f0a9db1ba2abcb2b2bb06643e2615d4d22f27daa316d5290e6cc99621c70913e: Status 404 returned error can't find the container with id f0a9db1ba2abcb2b2bb06643e2615d4d22f27daa316d5290e6cc99621c70913e
Jul 29 21:24:59.748188 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:59.748128    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9" event={"ID":"d514c30e-d844-4e08-9a90-a0f91329b77d","Type":"ContainerStarted","Data":"f0a9db1ba2abcb2b2bb06643e2615d4d22f27daa316d5290e6cc99621c70913e"}
Jul 29 21:24:59.846720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:59.846668    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:24:59.996793 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:24:59.996735    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:00.689802 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:00.689771    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:04.778216 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:04.778157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9" event={"ID":"d514c30e-d844-4e08-9a90-a0f91329b77d","Type":"ContainerStarted","Data":"e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8"}
Jul 29 21:25:04.778584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:04.778277    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:04.805545 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:04.805488    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9" podStartSLOduration=1.7173461209999998 podStartE2EDuration="6.805468103s" totalImagesPullingTime="5.088121982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:25:04.804410959 +0000 UTC m=+516.021188446" watchObservedRunningTime="2026-07-29 21:25:04.805468103 +0000 UTC m=+516.022245569"
Jul 29 21:25:05.215374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:05.215328    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:05.284287 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:05.284248    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:05.600398 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:05.600301    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:05.657556 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:05.657510    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:05.727015 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:05.726979    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7bvsf"
Jul 29 21:25:07.191595 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.191564    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"]
Jul 29 21:25:07.195961 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.195935    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.197238 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.197209    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:07.198292 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.198272    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 29 21:25:07.198397 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.198272    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 29 21:25:07.198397 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.198320    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ztd8p\""
Jul 29 21:25:07.205669 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.205649    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"]
Jul 29 21:25:07.307500 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.307451    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:07.362763 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.362729    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.362763 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.362766    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j489q\" (UniqueName: \"kubernetes.io/projected/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-kube-api-access-j489q\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.362959 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.362792    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.464387 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.464296    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.464387 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.464335    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j489q\" (UniqueName: \"kubernetes.io/projected/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-kube-api-access-j489q\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.464387 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.464361    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.465028 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.465009    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.466746 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.466716    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.472491 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.472463    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j489q\" (UniqueName: \"kubernetes.io/projected/b9e6aa38-fe24-4e94-b741-25bfaa10e60c-kube-api-access-j489q\") pod \"kuadrant-console-plugin-66b5774985-9sl6j\" (UID: \"b9e6aa38-fe24-4e94-b741-25bfaa10e60c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.506544 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.506504    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"
Jul 29 21:25:07.644288 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.644263    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j"]
Jul 29 21:25:07.646487 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:25:07.646455    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9e6aa38_fe24_4e94_b741_25bfaa10e60c.slice/crio-6ae63e018fb25bbc007e7538e16f1886dc9be6aed73ef56d3be79bb6ada397b1 WatchSource:0}: Error finding container 6ae63e018fb25bbc007e7538e16f1886dc9be6aed73ef56d3be79bb6ada397b1: Status 404 returned error can't find the container with id 6ae63e018fb25bbc007e7538e16f1886dc9be6aed73ef56d3be79bb6ada397b1
Jul 29 21:25:07.739938 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.739857    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5tk"
Jul 29 21:25:07.795350 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:07.795317    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j" event={"ID":"b9e6aa38-fe24-4e94-b741-25bfaa10e60c","Type":"ContainerStarted","Data":"6ae63e018fb25bbc007e7538e16f1886dc9be6aed73ef56d3be79bb6ada397b1"}
Jul 29 21:25:08.625833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:08.625794    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:08.725023 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:08.724982    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:12.773933 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:12.773889    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:12.875191 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:12.875130    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:15.786799 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:15.786766    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:17.678496 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.678462    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"]
Jul 29 21:25:17.679080 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.678688    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" containerName="manager" containerID="cri-o://e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8" gracePeriod=2
Jul 29 21:25:17.688943 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.688911    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"]
Jul 29 21:25:17.690958 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:17.690812    2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d514c30e-d844-4e08-9a90-a0f91329b77d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-prjc9\": pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:17.703061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.703034    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"]
Jul 29 21:25:17.703364 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.703282    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72" podUID="b93759e7-3910-4067-a59c-988ab7106482" containerName="manager" containerID="cri-o://cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a" gracePeriod=2
Jul 29 21:25:17.713756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.713724    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"]
Jul 29 21:25:17.714492 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.714466    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" containerName="manager"
Jul 29 21:25:17.714649 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.714633    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" containerName="manager"
Jul 29 21:25:17.714881 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.714859    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" containerName="manager"
Jul 29 21:25:17.718591 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.718574    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:17.726656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.726631    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"]
Jul 29 21:25:17.735537 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.735469    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"]
Jul 29 21:25:17.741260 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.741231    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"]
Jul 29 21:25:17.741625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.741600    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93759e7-3910-4067-a59c-988ab7106482" containerName="manager"
Jul 29 21:25:17.741625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.741623    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93759e7-3910-4067-a59c-988ab7106482" containerName="manager"
Jul 29 21:25:17.741804 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.741713    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93759e7-3910-4067-a59c-988ab7106482" containerName="manager"
Jul 29 21:25:17.744542 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:17.744503    2572 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b93759e7-3910-4067-a59c-988ab7106482\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-22r72\": pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:17.745014 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.744993    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:17.746472 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:17.746445    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:17.763070 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.763032    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"]
Jul 29 21:25:17.763394 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.763365    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1fd7b1d-2177-45ec-9834-3403b72948d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mh4qf\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:17.763550 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.763524    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh4tm\" (UniqueName: \"kubernetes.io/projected/b329228a-d4b1-41e5-9776-064d254597eb-kube-api-access-hh4tm\") pod \"limitador-operator-controller-manager-859d7b78bd-9bgn6\" (UID: \"b329228a-d4b1-41e5-9776-064d254597eb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:17.763624 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.763581    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzpj5\" (UniqueName: \"kubernetes.io/projected/f1fd7b1d-2177-45ec-9834-3403b72948d3-kube-api-access-vzpj5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mh4qf\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:17.776646 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:17.776578    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b93759e7-3910-4067-a59c-988ab7106482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:17.779341 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:17.779301    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:17.864777 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.864734    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1fd7b1d-2177-45ec-9834-3403b72948d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mh4qf\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:17.864978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.864834    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh4tm\" (UniqueName: \"kubernetes.io/projected/b329228a-d4b1-41e5-9776-064d254597eb-kube-api-access-hh4tm\") pod \"limitador-operator-controller-manager-859d7b78bd-9bgn6\" (UID: \"b329228a-d4b1-41e5-9776-064d254597eb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:17.864978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.864869    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzpj5\" (UniqueName: \"kubernetes.io/projected/f1fd7b1d-2177-45ec-9834-3403b72948d3-kube-api-access-vzpj5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mh4qf\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:17.865221 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.865196    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1fd7b1d-2177-45ec-9834-3403b72948d3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mh4qf\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:17.875201 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.875157    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh4tm\" (UniqueName: \"kubernetes.io/projected/b329228a-d4b1-41e5-9776-064d254597eb-kube-api-access-hh4tm\") pod \"limitador-operator-controller-manager-859d7b78bd-9bgn6\" (UID: \"b329228a-d4b1-41e5-9776-064d254597eb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:17.875339 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:17.875252    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzpj5\" (UniqueName: \"kubernetes.io/projected/f1fd7b1d-2177-45ec-9834-3403b72948d3-kube-api-access-vzpj5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mh4qf\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:18.105636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:18.105550    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:18.113199 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:18.113151    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:19.304963 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:19.304900    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:19.306962 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:19.306895    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b93759e7-3910-4067-a59c-988ab7106482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:23.593150 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:23.593063    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:23.698643 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:23.698601    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:28.782236 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:28.782164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:28.883799 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:28.883758    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:30.389523 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.389477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:30.460332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.460300    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:30.629298 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.629190    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"]
Jul 29 21:25:30.630974 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:25:30.630937    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb329228a_d4b1_41e5_9776_064d254597eb.slice/crio-de511287a15b62d10cf52f0c5b6f7164493414da5a08935406ec4b4a2861b534 WatchSource:0}: Error finding container de511287a15b62d10cf52f0c5b6f7164493414da5a08935406ec4b4a2861b534: Status 404 returned error can't find the container with id de511287a15b62d10cf52f0c5b6f7164493414da5a08935406ec4b4a2861b534
Jul 29 21:25:30.633869 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.633840    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:30.636213 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.636180    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b93759e7-3910-4067-a59c-988ab7106482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:30.637701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.637679    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:30.639721 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.639670    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b93759e7-3910-4067-a59c-988ab7106482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:30.641514 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.641482    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:30.660591 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.660569    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"]
Jul 29 21:25:30.662253 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:25:30.662232    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1fd7b1d_2177_45ec_9834_3403b72948d3.slice/crio-dd061b4ded81f6e904494c311f826de6e9b856dfc0ecbe1aef25e49fc9b70755 WatchSource:0}: Error finding container dd061b4ded81f6e904494c311f826de6e9b856dfc0ecbe1aef25e49fc9b70755: Status 404 returned error can't find the container with id dd061b4ded81f6e904494c311f826de6e9b856dfc0ecbe1aef25e49fc9b70755
Jul 29 21:25:30.699916 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.699895    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r9lr\" (UniqueName: \"kubernetes.io/projected/d514c30e-d844-4e08-9a90-a0f91329b77d-kube-api-access-6r9lr\") pod \"d514c30e-d844-4e08-9a90-a0f91329b77d\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") "
Jul 29 21:25:30.700046 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.699923    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d514c30e-d844-4e08-9a90-a0f91329b77d-extensions-socket-volume\") pod \"d514c30e-d844-4e08-9a90-a0f91329b77d\" (UID: \"d514c30e-d844-4e08-9a90-a0f91329b77d\") "
Jul 29 21:25:30.700046 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.699983    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn4sz\" (UniqueName: \"kubernetes.io/projected/b93759e7-3910-4067-a59c-988ab7106482-kube-api-access-dn4sz\") pod \"b93759e7-3910-4067-a59c-988ab7106482\" (UID: \"b93759e7-3910-4067-a59c-988ab7106482\") "
Jul 29 21:25:30.700627 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.700596    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d514c30e-d844-4e08-9a90-a0f91329b77d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d514c30e-d844-4e08-9a90-a0f91329b77d" (UID: "d514c30e-d844-4e08-9a90-a0f91329b77d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:25:30.702035 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.702013    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d514c30e-d844-4e08-9a90-a0f91329b77d-kube-api-access-6r9lr" (OuterVolumeSpecName: "kube-api-access-6r9lr") pod "d514c30e-d844-4e08-9a90-a0f91329b77d" (UID: "d514c30e-d844-4e08-9a90-a0f91329b77d"). InnerVolumeSpecName "kube-api-access-6r9lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:25:30.702504 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.702490    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93759e7-3910-4067-a59c-988ab7106482-kube-api-access-dn4sz" (OuterVolumeSpecName: "kube-api-access-dn4sz") pod "b93759e7-3910-4067-a59c-988ab7106482" (UID: "b93759e7-3910-4067-a59c-988ab7106482"). InnerVolumeSpecName "kube-api-access-dn4sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:25:30.801230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.801198    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn4sz\" (UniqueName: \"kubernetes.io/projected/b93759e7-3910-4067-a59c-988ab7106482-kube-api-access-dn4sz\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:25:30.801230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.801224    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r9lr\" (UniqueName: \"kubernetes.io/projected/d514c30e-d844-4e08-9a90-a0f91329b77d-kube-api-access-6r9lr\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:25:30.801230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.801234    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d514c30e-d844-4e08-9a90-a0f91329b77d-extensions-socket-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:25:30.937112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.937074    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf" event={"ID":"f1fd7b1d-2177-45ec-9834-3403b72948d3","Type":"ContainerStarted","Data":"33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632"}
Jul 29 21:25:30.937327 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.937159    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf" event={"ID":"f1fd7b1d-2177-45ec-9834-3403b72948d3","Type":"ContainerStarted","Data":"dd061b4ded81f6e904494c311f826de6e9b856dfc0ecbe1aef25e49fc9b70755"}
Jul 29 21:25:30.937327 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.937217    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:30.938815 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.938788    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6" event={"ID":"b329228a-d4b1-41e5-9776-064d254597eb","Type":"ContainerStarted","Data":"9a90d27fbc995204d4b8067f4870602f5b5b2ba61594e11c99faf64bd7e2f56c"}
Jul 29 21:25:30.938931 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.938844    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6" event={"ID":"b329228a-d4b1-41e5-9776-064d254597eb","Type":"ContainerStarted","Data":"de511287a15b62d10cf52f0c5b6f7164493414da5a08935406ec4b4a2861b534"}
Jul 29 21:25:30.938931 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.938873    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:30.939251 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.939215    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:30.940049 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.940025    2572 generic.go:356] "Generic (PLEG): container finished" podID="d514c30e-d844-4e08-9a90-a0f91329b77d" containerID="e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8" exitCode=0
Jul 29 21:25:30.940099 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.940075    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:30.940134 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.940123    2572 scope.go:117] "RemoveContainer" containerID="e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8"
Jul 29 21:25:30.941464 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.941440    2572 generic.go:356] "Generic (PLEG): container finished" podID="b93759e7-3910-4067-a59c-988ab7106482" containerID="cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a" exitCode=0
Jul 29 21:25:30.941590 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.941508    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:30.943273 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.943249    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j" event={"ID":"b9e6aa38-fe24-4e94-b741-25bfaa10e60c","Type":"ContainerStarted","Data":"4a0e871e8a373891160d1c8df516e3aaced2551ad7598894b6117045cdb228ec"}
Jul 29 21:25:30.949513 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.949492    2572 scope.go:117] "RemoveContainer" containerID="e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8"
Jul 29 21:25:30.949804 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.949784    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8\": container with ID starting with e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8 not found: ID does not exist" containerID="e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8"
Jul 29 21:25:30.949862 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.949813    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8"} err="failed to get container status \"e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8\": rpc error: code = NotFound desc = could not find container \"e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8\": container with ID starting with e18897680e73de9459dd95a4bf576bc402830087c1cf1d5dc1133dfd2cd031c8 not found: ID does not exist"
Jul 29 21:25:30.949862 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.949830    2572 scope.go:117] "RemoveContainer" containerID="cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a"
Jul 29 21:25:30.958034 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.958018    2572 scope.go:117] "RemoveContainer" containerID="cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a"
Jul 29 21:25:30.958291 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.958272    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a\": container with ID starting with cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a not found: ID does not exist" containerID="cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a"
Jul 29 21:25:30.958342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.958297    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a"} err="failed to get container status \"cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a\": rpc error: code = NotFound desc = could not find container \"cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a\": container with ID starting with cbe461dacb1ac471aaa432063c3ae1801eaf170d2ec172402ad00803141ab74a not found: ID does not exist"
Jul 29 21:25:30.963483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.963445    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf" podStartSLOduration=13.963431918 podStartE2EDuration="13.963431918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:25:30.961856823 +0000 UTC m=+542.178634285" watchObservedRunningTime="2026-07-29 21:25:30.963431918 +0000 UTC m=+542.180209380"
Jul 29 21:25:30.963830 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.963804    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b93759e7-3910-4067-a59c-988ab7106482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:30.965586 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.965563    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-22r72\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b93759e7-3910-4067-a59c-988ab7106482" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-22r72"
Jul 29 21:25:30.984607 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:30.984567    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9sl6j" podStartSLOduration=1.068604191 podStartE2EDuration="23.98455259s" totalImagesPullingTime="22.915948399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:25:30.983194741 +0000 UTC m=+542.199972205" watchObservedRunningTime="2026-07-29 21:25:30.98455259 +0000 UTC m=+542.201330052"
Jul 29 21:25:30.985309 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:30.985282    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prjc9\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prjc9"
Jul 29 21:25:31.006213 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:31.006153    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6" podStartSLOduration=14.006138642 podStartE2EDuration="14.006138642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:25:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:25:31.004159082 +0000 UTC m=+542.220936543" watchObservedRunningTime="2026-07-29 21:25:31.006138642 +0000 UTC m=+542.222916104"
Jul 29 21:25:31.304224 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:31.304127    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93759e7-3910-4067-a59c-988ab7106482" path="/var/lib/kubelet/pods/b93759e7-3910-4067-a59c-988ab7106482/volumes"
Jul 29 21:25:31.304529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:31.304515    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d514c30e-d844-4e08-9a90-a0f91329b77d" path="/var/lib/kubelet/pods/d514c30e-d844-4e08-9a90-a0f91329b77d/volumes"
Jul 29 21:25:33.577345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.577307    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"]
Jul 29 21:25:33.577832 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.577570    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf" podUID="f1fd7b1d-2177-45ec-9834-3403b72948d3" containerName="manager" containerID="cri-o://33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632" gracePeriod=10
Jul 29 21:25:33.832497 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.832435    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:33.893690 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.893650    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"]
Jul 29 21:25:33.894067 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.894054    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1fd7b1d-2177-45ec-9834-3403b72948d3" containerName="manager"
Jul 29 21:25:33.894121 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.894070    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1fd7b1d-2177-45ec-9834-3403b72948d3" containerName="manager"
Jul 29 21:25:33.894156 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.894131    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1fd7b1d-2177-45ec-9834-3403b72948d3" containerName="manager"
Jul 29 21:25:33.897062 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.897046    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:33.920399 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.920361    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"]
Jul 29 21:25:33.929337 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.929303    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1fd7b1d-2177-45ec-9834-3403b72948d3-extensions-socket-volume\") pod \"f1fd7b1d-2177-45ec-9834-3403b72948d3\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") "
Jul 29 21:25:33.929529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.929357    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzpj5\" (UniqueName: \"kubernetes.io/projected/f1fd7b1d-2177-45ec-9834-3403b72948d3-kube-api-access-vzpj5\") pod \"f1fd7b1d-2177-45ec-9834-3403b72948d3\" (UID: \"f1fd7b1d-2177-45ec-9834-3403b72948d3\") "
Jul 29 21:25:33.929772 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.929744    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1fd7b1d-2177-45ec-9834-3403b72948d3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f1fd7b1d-2177-45ec-9834-3403b72948d3" (UID: "f1fd7b1d-2177-45ec-9834-3403b72948d3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:25:33.931610 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.931571    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1fd7b1d-2177-45ec-9834-3403b72948d3-kube-api-access-vzpj5" (OuterVolumeSpecName: "kube-api-access-vzpj5") pod "f1fd7b1d-2177-45ec-9834-3403b72948d3" (UID: "f1fd7b1d-2177-45ec-9834-3403b72948d3"). InnerVolumeSpecName "kube-api-access-vzpj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:25:33.962290 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.962252    2572 generic.go:356] "Generic (PLEG): container finished" podID="f1fd7b1d-2177-45ec-9834-3403b72948d3" containerID="33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632" exitCode=0
Jul 29 21:25:33.962478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.962298    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"
Jul 29 21:25:33.962478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.962321    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf" event={"ID":"f1fd7b1d-2177-45ec-9834-3403b72948d3","Type":"ContainerDied","Data":"33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632"}
Jul 29 21:25:33.962478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.962365    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf" event={"ID":"f1fd7b1d-2177-45ec-9834-3403b72948d3","Type":"ContainerDied","Data":"dd061b4ded81f6e904494c311f826de6e9b856dfc0ecbe1aef25e49fc9b70755"}
Jul 29 21:25:33.962478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.962387    2572 scope.go:117] "RemoveContainer" containerID="33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632"
Jul 29 21:25:33.972640 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.972620    2572 scope.go:117] "RemoveContainer" containerID="33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632"
Jul 29 21:25:33.972960 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:25:33.972938    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632\": container with ID starting with 33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632 not found: ID does not exist" containerID="33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632"
Jul 29 21:25:33.973001 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.972968    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632"} err="failed to get container status \"33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632\": rpc error: code = NotFound desc = could not find container \"33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632\": container with ID starting with 33cf444c2c7daeca3edb980e91fbf8dec6e625734fd6cf2c5a41d77fc34b4632 not found: ID does not exist"
Jul 29 21:25:33.991990 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.991964    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"]
Jul 29 21:25:33.998708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:33.998679    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mh4qf"]
Jul 29 21:25:34.030113 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.030084    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lz5p\" (UniqueName: \"kubernetes.io/projected/c0327482-5999-42f7-a015-9a30d10f6ab9-kube-api-access-5lz5p\") pod \"kuadrant-operator-controller-manager-86878c769f-g7mqq\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.030255 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.030157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0327482-5999-42f7-a015-9a30d10f6ab9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g7mqq\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.030255 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.030241    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1fd7b1d-2177-45ec-9834-3403b72948d3-extensions-socket-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:25:34.030255 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.030253    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzpj5\" (UniqueName: \"kubernetes.io/projected/f1fd7b1d-2177-45ec-9834-3403b72948d3-kube-api-access-vzpj5\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:25:34.131571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.131470    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lz5p\" (UniqueName: \"kubernetes.io/projected/c0327482-5999-42f7-a015-9a30d10f6ab9-kube-api-access-5lz5p\") pod \"kuadrant-operator-controller-manager-86878c769f-g7mqq\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.131765 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.131565    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0327482-5999-42f7-a015-9a30d10f6ab9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g7mqq\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.131982 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.131957    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0327482-5999-42f7-a015-9a30d10f6ab9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g7mqq\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.141109 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.141070    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lz5p\" (UniqueName: \"kubernetes.io/projected/c0327482-5999-42f7-a015-9a30d10f6ab9-kube-api-access-5lz5p\") pod \"kuadrant-operator-controller-manager-86878c769f-g7mqq\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.207818 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.207781    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.366739 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.366712    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"]
Jul 29 21:25:34.368964 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:25:34.368936    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0327482_5999_42f7_a015_9a30d10f6ab9.slice/crio-cf5e729ada9760fd316591f65dccc70dd5af80f97e7da7951c050642c9e2a096 WatchSource:0}: Error finding container cf5e729ada9760fd316591f65dccc70dd5af80f97e7da7951c050642c9e2a096: Status 404 returned error can't find the container with id cf5e729ada9760fd316591f65dccc70dd5af80f97e7da7951c050642c9e2a096
Jul 29 21:25:34.975060 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.975003    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq" event={"ID":"c0327482-5999-42f7-a015-9a30d10f6ab9","Type":"ContainerStarted","Data":"9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b"}
Jul 29 21:25:34.975541 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.975129    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq" event={"ID":"c0327482-5999-42f7-a015-9a30d10f6ab9","Type":"ContainerStarted","Data":"cf5e729ada9760fd316591f65dccc70dd5af80f97e7da7951c050642c9e2a096"}
Jul 29 21:25:34.975586 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.975559    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:34.999475 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:34.999425    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq" podStartSLOduration=1.999407786 podStartE2EDuration="1.999407786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:25:34.998445832 +0000 UTC m=+546.215223294" watchObservedRunningTime="2026-07-29 21:25:34.999407786 +0000 UTC m=+546.216185249"
Jul 29 21:25:35.298809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:35.298715    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:35.304688 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:35.304662    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1fd7b1d-2177-45ec-9834-3403b72948d3" path="/var/lib/kubelet/pods/f1fd7b1d-2177-45ec-9834-3403b72948d3/volumes"
Jul 29 21:25:35.404729 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:35.404692    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:35.498889 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:35.498853    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:41.953546 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:41.953512    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-9bgn6"
Jul 29 21:25:45.981867 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:45.981833    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:25:50.106088 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:50.106050    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:25:50.240779 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:25:50.240744    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:05.141828 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.141792    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:05.246383 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.246352    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:05.347651 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.347614    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:05.612798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.612758    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:26:05.708064 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.708029    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:26:05.708064 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.708067    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:26:05.708290 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.708145    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.710612 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.710589    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 21:26:05.815529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.815494    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5t6x\" (UniqueName: \"kubernetes.io/projected/897dd401-eb41-4c00-be54-2d5427c7ee55-kube-api-access-g5t6x\") pod \"limitador-limitador-54d6d988d5-b4szp\" (UID: \"897dd401-eb41-4c00-be54-2d5427c7ee55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.815710 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.815553    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/897dd401-eb41-4c00-be54-2d5427c7ee55-config-file\") pod \"limitador-limitador-54d6d988d5-b4szp\" (UID: \"897dd401-eb41-4c00-be54-2d5427c7ee55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.916543 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.916461    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5t6x\" (UniqueName: \"kubernetes.io/projected/897dd401-eb41-4c00-be54-2d5427c7ee55-kube-api-access-g5t6x\") pod \"limitador-limitador-54d6d988d5-b4szp\" (UID: \"897dd401-eb41-4c00-be54-2d5427c7ee55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.916543 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.916523    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/897dd401-eb41-4c00-be54-2d5427c7ee55-config-file\") pod \"limitador-limitador-54d6d988d5-b4szp\" (UID: \"897dd401-eb41-4c00-be54-2d5427c7ee55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.917142 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.917123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/897dd401-eb41-4c00-be54-2d5427c7ee55-config-file\") pod \"limitador-limitador-54d6d988d5-b4szp\" (UID: \"897dd401-eb41-4c00-be54-2d5427c7ee55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.925099 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.925066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5t6x\" (UniqueName: \"kubernetes.io/projected/897dd401-eb41-4c00-be54-2d5427c7ee55-kube-api-access-g5t6x\") pod \"limitador-limitador-54d6d988d5-b4szp\" (UID: \"897dd401-eb41-4c00-be54-2d5427c7ee55\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:05.929444 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.929418    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-qfvd9"]
Jul 29 21:26:05.933408 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.933382    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:05.935941 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.935920    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nb2b2\""
Jul 29 21:26:05.941563 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.941539    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qfvd9"]
Jul 29 21:26:05.959845 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:05.959820    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:06.018001 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.017968    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg97x\" (UniqueName: \"kubernetes.io/projected/a86d6212-3651-4801-9a5e-e037b94a1908-kube-api-access-rg97x\") pod \"authorino-58cd955675-qfvd9\" (UID: \"a86d6212-3651-4801-9a5e-e037b94a1908\") " pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:06.018756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.018735    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:06.060661 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.060622    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:06.119256 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.119229    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg97x\" (UniqueName: \"kubernetes.io/projected/a86d6212-3651-4801-9a5e-e037b94a1908-kube-api-access-rg97x\") pod \"authorino-58cd955675-qfvd9\" (UID: \"a86d6212-3651-4801-9a5e-e037b94a1908\") " pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:06.133004 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.132971    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg97x\" (UniqueName: \"kubernetes.io/projected/a86d6212-3651-4801-9a5e-e037b94a1908-kube-api-access-rg97x\") pod \"authorino-58cd955675-qfvd9\" (UID: \"a86d6212-3651-4801-9a5e-e037b94a1908\") " pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:06.158608 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.158583    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:26:06.161343 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:26:06.161316    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod897dd401_eb41_4c00_be54_2d5427c7ee55.slice/crio-6491c5eb5d6b9b3fea9534e489c5965caf5345f0a7ba8517ece29d1a7afd588b WatchSource:0}: Error finding container 6491c5eb5d6b9b3fea9534e489c5965caf5345f0a7ba8517ece29d1a7afd588b: Status 404 returned error can't find the container with id 6491c5eb5d6b9b3fea9534e489c5965caf5345f0a7ba8517ece29d1a7afd588b
Jul 29 21:26:06.224421 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.224386    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-vwknj"]
Jul 29 21:26:06.229398 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.229378    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:06.234723 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.234699    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vwknj"]
Jul 29 21:26:06.243258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.243238    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:06.321710 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.321671    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fvcx\" (UniqueName: \"kubernetes.io/projected/923d8787-4ae5-4f80-a2b4-25fd02641543-kube-api-access-6fvcx\") pod \"authorino-75d556bd77-vwknj\" (UID: \"923d8787-4ae5-4f80-a2b4-25fd02641543\") " pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:06.373959 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.373935    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qfvd9"]
Jul 29 21:26:06.376033 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:26:06.376004    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda86d6212_3651_4801_9a5e_e037b94a1908.slice/crio-c05782b64074a8a6c65086217d2c282d850bd988635deba4952471233a51f268 WatchSource:0}: Error finding container c05782b64074a8a6c65086217d2c282d850bd988635deba4952471233a51f268: Status 404 returned error can't find the container with id c05782b64074a8a6c65086217d2c282d850bd988635deba4952471233a51f268
Jul 29 21:26:06.423017 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.422982    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fvcx\" (UniqueName: \"kubernetes.io/projected/923d8787-4ae5-4f80-a2b4-25fd02641543-kube-api-access-6fvcx\") pod \"authorino-75d556bd77-vwknj\" (UID: \"923d8787-4ae5-4f80-a2b4-25fd02641543\") " pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:06.432048 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.432026    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fvcx\" (UniqueName: \"kubernetes.io/projected/923d8787-4ae5-4f80-a2b4-25fd02641543-kube-api-access-6fvcx\") pod \"authorino-75d556bd77-vwknj\" (UID: \"923d8787-4ae5-4f80-a2b4-25fd02641543\") " pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:06.540107 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.540024    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:06.690612 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:06.690548    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vwknj"]
Jul 29 21:26:06.692798 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:26:06.692774    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod923d8787_4ae5_4f80_a2b4_25fd02641543.slice/crio-dd71530502277598011ea450ad2a5b95b56039f747a3df68061d145df31da426 WatchSource:0}: Error finding container dd71530502277598011ea450ad2a5b95b56039f747a3df68061d145df31da426: Status 404 returned error can't find the container with id dd71530502277598011ea450ad2a5b95b56039f747a3df68061d145df31da426
Jul 29 21:26:07.152344 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:07.152248    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vwknj" event={"ID":"923d8787-4ae5-4f80-a2b4-25fd02641543","Type":"ContainerStarted","Data":"dd71530502277598011ea450ad2a5b95b56039f747a3df68061d145df31da426"}
Jul 29 21:26:07.159606 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:07.159568    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qfvd9" event={"ID":"a86d6212-3651-4801-9a5e-e037b94a1908","Type":"ContainerStarted","Data":"c05782b64074a8a6c65086217d2c282d850bd988635deba4952471233a51f268"}
Jul 29 21:26:07.163729 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:07.163658    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp" event={"ID":"897dd401-eb41-4c00-be54-2d5427c7ee55","Type":"ContainerStarted","Data":"6491c5eb5d6b9b3fea9534e489c5965caf5345f0a7ba8517ece29d1a7afd588b"}
Jul 29 21:26:12.204396 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.204359    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vwknj" event={"ID":"923d8787-4ae5-4f80-a2b4-25fd02641543","Type":"ContainerStarted","Data":"34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb"}
Jul 29 21:26:12.205924 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.205897    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qfvd9" event={"ID":"a86d6212-3651-4801-9a5e-e037b94a1908","Type":"ContainerStarted","Data":"33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81"}
Jul 29 21:26:12.207495 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.207469    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp" event={"ID":"897dd401-eb41-4c00-be54-2d5427c7ee55","Type":"ContainerStarted","Data":"d57a1a4466856ee6d52e50326bdf658c9dec3be6ca0c6d33e191ddbe946c1c02"}
Jul 29 21:26:12.207566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.207550    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:12.222230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.222187    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-vwknj" podStartSLOduration=1.550562633 podStartE2EDuration="6.222156321s" totalImagesPullingTime="4.671593688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:12.220971242 +0000 UTC m=+583.437748705" watchObservedRunningTime="2026-07-29 21:26:12.222156321 +0000 UTC m=+583.438933785"
Jul 29 21:26:12.243112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.243056    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp" podStartSLOduration=2.032434271 podStartE2EDuration="7.24303867s" totalImagesPullingTime="5.210604399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:12.241068466 +0000 UTC m=+583.457845927" watchObservedRunningTime="2026-07-29 21:26:12.24303867 +0000 UTC m=+583.459816131"
Jul 29 21:26:12.254517 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.254484    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qfvd9"]
Jul 29 21:26:12.270761 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:12.270715    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-qfvd9" podStartSLOduration=2.323214909 podStartE2EDuration="7.27070019s" totalImagesPullingTime="4.947485281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:12.269155714 +0000 UTC m=+583.485933177" watchObservedRunningTime="2026-07-29 21:26:12.27070019 +0000 UTC m=+583.487477652"
Jul 29 21:26:14.217596 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.217555    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-qfvd9" podUID="a86d6212-3651-4801-9a5e-e037b94a1908" containerName="authorino" containerID="cri-o://33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81" gracePeriod=30
Jul 29 21:26:14.331566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.331536    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:14.410819 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.410775    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:14.468263 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.468210    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:14.604588 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.604549    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg97x\" (UniqueName: \"kubernetes.io/projected/a86d6212-3651-4801-9a5e-e037b94a1908-kube-api-access-rg97x\") pod \"a86d6212-3651-4801-9a5e-e037b94a1908\" (UID: \"a86d6212-3651-4801-9a5e-e037b94a1908\") "
Jul 29 21:26:14.606584 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.606552    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a86d6212-3651-4801-9a5e-e037b94a1908-kube-api-access-rg97x" (OuterVolumeSpecName: "kube-api-access-rg97x") pod "a86d6212-3651-4801-9a5e-e037b94a1908" (UID: "a86d6212-3651-4801-9a5e-e037b94a1908"). InnerVolumeSpecName "kube-api-access-rg97x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:26:14.705331 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:14.705293    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg97x\" (UniqueName: \"kubernetes.io/projected/a86d6212-3651-4801-9a5e-e037b94a1908-kube-api-access-rg97x\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:26:15.229721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.229686    2572 generic.go:356] "Generic (PLEG): container finished" podID="a86d6212-3651-4801-9a5e-e037b94a1908" containerID="33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81" exitCode=0
Jul 29 21:26:15.230138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.229743    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qfvd9"
Jul 29 21:26:15.230138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.229773    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qfvd9" event={"ID":"a86d6212-3651-4801-9a5e-e037b94a1908","Type":"ContainerDied","Data":"33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81"}
Jul 29 21:26:15.230138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.229816    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qfvd9" event={"ID":"a86d6212-3651-4801-9a5e-e037b94a1908","Type":"ContainerDied","Data":"c05782b64074a8a6c65086217d2c282d850bd988635deba4952471233a51f268"}
Jul 29 21:26:15.230138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.229833    2572 scope.go:117] "RemoveContainer" containerID="33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81"
Jul 29 21:26:15.239478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.239462    2572 scope.go:117] "RemoveContainer" containerID="33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81"
Jul 29 21:26:15.239741 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:26:15.239723    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81\": container with ID starting with 33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81 not found: ID does not exist" containerID="33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81"
Jul 29 21:26:15.239788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.239747    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81"} err="failed to get container status \"33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81\": rpc error: code = NotFound desc = could not find container \"33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81\": container with ID starting with 33581ade2922f397cc789d948e3e08b933cd0a9117a65951696f02278370be81 not found: ID does not exist"
Jul 29 21:26:15.256299 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.256280    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qfvd9"]
Jul 29 21:26:15.259507 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.259484    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-qfvd9"]
Jul 29 21:26:15.304397 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:15.304372    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a86d6212-3651-4801-9a5e-e037b94a1908" path="/var/lib/kubelet/pods/a86d6212-3651-4801-9a5e-e037b94a1908/volumes"
Jul 29 21:26:23.213676 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:23.213639    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-b4szp"
Jul 29 21:26:28.369749 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:28.369708    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:28.498599 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:28.498560    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:28.845691 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:28.845658    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:41.190737 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.190684    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:41.426195 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.426148    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbpm"]
Jul 29 21:26:41.426603 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.426587    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a86d6212-3651-4801-9a5e-e037b94a1908" containerName="authorino"
Jul 29 21:26:41.426650 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.426605    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a86d6212-3651-4801-9a5e-e037b94a1908" containerName="authorino"
Jul 29 21:26:41.426685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.426672    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a86d6212-3651-4801-9a5e-e037b94a1908" containerName="authorino"
Jul 29 21:26:41.430032 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.430007    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:41.438855 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.438826    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbpm"]
Jul 29 21:26:41.445878 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.445813    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b9f8\" (UniqueName: \"kubernetes.io/projected/26dfe913-b90a-4069-92bc-0144a2baa7be-kube-api-access-8b9f8\") pod \"authorino-5d4b67c446-jjbpm\" (UID: \"26dfe913-b90a-4069-92bc-0144a2baa7be\") " pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:41.546913 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.546855    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b9f8\" (UniqueName: \"kubernetes.io/projected/26dfe913-b90a-4069-92bc-0144a2baa7be-kube-api-access-8b9f8\") pod \"authorino-5d4b67c446-jjbpm\" (UID: \"26dfe913-b90a-4069-92bc-0144a2baa7be\") " pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:41.557240 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.557205    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b9f8\" (UniqueName: \"kubernetes.io/projected/26dfe913-b90a-4069-92bc-0144a2baa7be-kube-api-access-8b9f8\") pod \"authorino-5d4b67c446-jjbpm\" (UID: \"26dfe913-b90a-4069-92bc-0144a2baa7be\") " pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:41.678954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.678919    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbpm"]
Jul 29 21:26:41.679220 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.679207    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:41.722881 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.722794    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-684d965f97-hztjn"]
Jul 29 21:26:41.727552 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.727533    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:41.738608 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.738540    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-684d965f97-hztjn"]
Jul 29 21:26:41.748633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.748523    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rb27\" (UniqueName: \"kubernetes.io/projected/8304e7a6-03e3-4e13-881e-dd2e7089d661-kube-api-access-4rb27\") pod \"authorino-684d965f97-hztjn\" (UID: \"8304e7a6-03e3-4e13-881e-dd2e7089d661\") " pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:41.835489 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.835453    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbpm"]
Jul 29 21:26:41.836352 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:26:41.836325    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26dfe913_b90a_4069_92bc_0144a2baa7be.slice/crio-a3311d00c9b66308ea092dc3f98d8df1961163a5c13963db1bf84bef8c0bfcff WatchSource:0}: Error finding container a3311d00c9b66308ea092dc3f98d8df1961163a5c13963db1bf84bef8c0bfcff: Status 404 returned error can't find the container with id a3311d00c9b66308ea092dc3f98d8df1961163a5c13963db1bf84bef8c0bfcff
Jul 29 21:26:41.849624 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.849589    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rb27\" (UniqueName: \"kubernetes.io/projected/8304e7a6-03e3-4e13-881e-dd2e7089d661-kube-api-access-4rb27\") pod \"authorino-684d965f97-hztjn\" (UID: \"8304e7a6-03e3-4e13-881e-dd2e7089d661\") " pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:41.864253 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.864218    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rb27\" (UniqueName: \"kubernetes.io/projected/8304e7a6-03e3-4e13-881e-dd2e7089d661-kube-api-access-4rb27\") pod \"authorino-684d965f97-hztjn\" (UID: \"8304e7a6-03e3-4e13-881e-dd2e7089d661\") " pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:41.995911 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.995822    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-684d965f97-hztjn"]
Jul 29 21:26:41.996083 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:41.996071    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:42.047021 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.046889    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5c77c8474f-bbvxg"]
Jul 29 21:26:42.052345 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.052321    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.057319 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.057299    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 21:26:42.066952 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.066928    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c77c8474f-bbvxg"]
Jul 29 21:26:42.135936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.135909    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-684d965f97-hztjn"]
Jul 29 21:26:42.137612 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:26:42.137586    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8304e7a6_03e3_4e13_881e_dd2e7089d661.slice/crio-5a09edaa4a40734764a36d7c3d79d279bced8bfdb2e4c20b71149e00722524e9 WatchSource:0}: Error finding container 5a09edaa4a40734764a36d7c3d79d279bced8bfdb2e4c20b71149e00722524e9: Status 404 returned error can't find the container with id 5a09edaa4a40734764a36d7c3d79d279bced8bfdb2e4c20b71149e00722524e9
Jul 29 21:26:42.151894 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.151868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-tls-cert\") pod \"authorino-5c77c8474f-bbvxg\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") " pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.152000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.151948    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqj7w\" (UniqueName: \"kubernetes.io/projected/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-kube-api-access-vqj7w\") pod \"authorino-5c77c8474f-bbvxg\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") " pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.253031 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.252978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-tls-cert\") pod \"authorino-5c77c8474f-bbvxg\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") " pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.253349 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.253037    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqj7w\" (UniqueName: \"kubernetes.io/projected/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-kube-api-access-vqj7w\") pod \"authorino-5c77c8474f-bbvxg\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") " pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.255671 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.255644    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-tls-cert\") pod \"authorino-5c77c8474f-bbvxg\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") " pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.264049 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.264020    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqj7w\" (UniqueName: \"kubernetes.io/projected/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-kube-api-access-vqj7w\") pod \"authorino-5c77c8474f-bbvxg\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") " pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.365331 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.365292    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:26:42.371801 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.371761    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684d965f97-hztjn" event={"ID":"8304e7a6-03e3-4e13-881e-dd2e7089d661","Type":"ContainerStarted","Data":"5a09edaa4a40734764a36d7c3d79d279bced8bfdb2e4c20b71149e00722524e9"}
Jul 29 21:26:42.373436 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.373409    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbpm" event={"ID":"26dfe913-b90a-4069-92bc-0144a2baa7be","Type":"ContainerStarted","Data":"6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2"}
Jul 29 21:26:42.373573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.373427    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jjbpm" podUID="26dfe913-b90a-4069-92bc-0144a2baa7be" containerName="authorino" containerID="cri-o://6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2" gracePeriod=30
Jul 29 21:26:42.373573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.373446    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbpm" event={"ID":"26dfe913-b90a-4069-92bc-0144a2baa7be","Type":"ContainerStarted","Data":"a3311d00c9b66308ea092dc3f98d8df1961163a5c13963db1bf84bef8c0bfcff"}
Jul 29 21:26:42.391298 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.391260    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jjbpm" podStartSLOduration=1.040844374 podStartE2EDuration="1.39124474s" totalImagesPullingTime="350.400366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:42.388608226 +0000 UTC m=+613.605385689" watchObservedRunningTime="2026-07-29 21:26:42.39124474 +0000 UTC m=+613.608022232"
Jul 29 21:26:42.528106 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.528074    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5c77c8474f-bbvxg"]
Jul 29 21:26:42.529790 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:26:42.529760    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a9c1fe9_e48f_47b5_b355_2704f25f46d8.slice/crio-4e188b26ffc5043265ded901801356aa2566fa92079f658a7f3e19e0702b8b9d WatchSource:0}: Error finding container 4e188b26ffc5043265ded901801356aa2566fa92079f658a7f3e19e0702b8b9d: Status 404 returned error can't find the container with id 4e188b26ffc5043265ded901801356aa2566fa92079f658a7f3e19e0702b8b9d
Jul 29 21:26:42.629984 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.629947    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:42.756991 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.756952    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b9f8\" (UniqueName: \"kubernetes.io/projected/26dfe913-b90a-4069-92bc-0144a2baa7be-kube-api-access-8b9f8\") pod \"26dfe913-b90a-4069-92bc-0144a2baa7be\" (UID: \"26dfe913-b90a-4069-92bc-0144a2baa7be\") "
Jul 29 21:26:42.759009 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.758974    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26dfe913-b90a-4069-92bc-0144a2baa7be-kube-api-access-8b9f8" (OuterVolumeSpecName: "kube-api-access-8b9f8") pod "26dfe913-b90a-4069-92bc-0144a2baa7be" (UID: "26dfe913-b90a-4069-92bc-0144a2baa7be"). InnerVolumeSpecName "kube-api-access-8b9f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:26:42.858024 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:42.857935    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b9f8\" (UniqueName: \"kubernetes.io/projected/26dfe913-b90a-4069-92bc-0144a2baa7be-kube-api-access-8b9f8\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:26:43.275561 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.275521    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:43.385570 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.385528    2572 generic.go:356] "Generic (PLEG): container finished" podID="26dfe913-b90a-4069-92bc-0144a2baa7be" containerID="6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2" exitCode=0
Jul 29 21:26:43.385751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.385582    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jjbpm"
Jul 29 21:26:43.385751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.385588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbpm" event={"ID":"26dfe913-b90a-4069-92bc-0144a2baa7be","Type":"ContainerDied","Data":"6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2"}
Jul 29 21:26:43.385751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.385643    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jjbpm" event={"ID":"26dfe913-b90a-4069-92bc-0144a2baa7be","Type":"ContainerDied","Data":"a3311d00c9b66308ea092dc3f98d8df1961163a5c13963db1bf84bef8c0bfcff"}
Jul 29 21:26:43.385751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.385665    2572 scope.go:117] "RemoveContainer" containerID="6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2"
Jul 29 21:26:43.387735 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.387709    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77c8474f-bbvxg" event={"ID":"0a9c1fe9-e48f-47b5-b355-2704f25f46d8","Type":"ContainerStarted","Data":"bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d"}
Jul 29 21:26:43.388006 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.387745    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77c8474f-bbvxg" event={"ID":"0a9c1fe9-e48f-47b5-b355-2704f25f46d8","Type":"ContainerStarted","Data":"4e188b26ffc5043265ded901801356aa2566fa92079f658a7f3e19e0702b8b9d"}
Jul 29 21:26:43.389551 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.389526    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684d965f97-hztjn" event={"ID":"8304e7a6-03e3-4e13-881e-dd2e7089d661","Type":"ContainerStarted","Data":"105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193"}
Jul 29 21:26:43.389667 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.389641    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-684d965f97-hztjn" podUID="8304e7a6-03e3-4e13-881e-dd2e7089d661" containerName="authorino" containerID="cri-o://105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193" gracePeriod=30
Jul 29 21:26:43.395605 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.395587    2572 scope.go:117] "RemoveContainer" containerID="6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2"
Jul 29 21:26:43.395866 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:26:43.395844    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2\": container with ID starting with 6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2 not found: ID does not exist" containerID="6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2"
Jul 29 21:26:43.395911 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.395876    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2"} err="failed to get container status \"6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2\": rpc error: code = NotFound desc = could not find container \"6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2\": container with ID starting with 6a69ebe0004cbec7ffd5cec2f293097a1c4375fb7b5bb90bcb7d0ad3928598a2 not found: ID does not exist"
Jul 29 21:26:43.403754 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.403731    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbpm"]
Jul 29 21:26:43.408839 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.408820    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jjbpm"]
Jul 29 21:26:43.423884 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.423834    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-684d965f97-hztjn" podStartSLOduration=1.372915447 podStartE2EDuration="2.423817939s" totalImagesPullingTime="1.050902492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:43.421820642 +0000 UTC m=+614.638598105" watchObservedRunningTime="2026-07-29 21:26:43.423817939 +0000 UTC m=+614.640595401"
Jul 29 21:26:43.438451 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.438396    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5c77c8474f-bbvxg" podStartSLOduration=0.950645112 podStartE2EDuration="1.438381172s" totalImagesPullingTime="487.73606ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:26:43.436563766 +0000 UTC m=+614.653341230" watchObservedRunningTime="2026-07-29 21:26:43.438381172 +0000 UTC m=+614.655158662"
Jul 29 21:26:43.467272 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.467193    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vwknj"]
Jul 29 21:26:43.467403 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.467358    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-vwknj" podUID="923d8787-4ae5-4f80-a2b4-25fd02641543" containerName="authorino" containerID="cri-o://34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb" gracePeriod=30
Jul 29 21:26:43.743491 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.743465    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:43.746402 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.746380    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:43.764366 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.764329    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rb27\" (UniqueName: \"kubernetes.io/projected/8304e7a6-03e3-4e13-881e-dd2e7089d661-kube-api-access-4rb27\") pod \"8304e7a6-03e3-4e13-881e-dd2e7089d661\" (UID: \"8304e7a6-03e3-4e13-881e-dd2e7089d661\") "
Jul 29 21:26:43.764551 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.764420    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fvcx\" (UniqueName: \"kubernetes.io/projected/923d8787-4ae5-4f80-a2b4-25fd02641543-kube-api-access-6fvcx\") pod \"923d8787-4ae5-4f80-a2b4-25fd02641543\" (UID: \"923d8787-4ae5-4f80-a2b4-25fd02641543\") "
Jul 29 21:26:43.766622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.766559    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8304e7a6-03e3-4e13-881e-dd2e7089d661-kube-api-access-4rb27" (OuterVolumeSpecName: "kube-api-access-4rb27") pod "8304e7a6-03e3-4e13-881e-dd2e7089d661" (UID: "8304e7a6-03e3-4e13-881e-dd2e7089d661"). InnerVolumeSpecName "kube-api-access-4rb27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:26:43.766622 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.766569    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/923d8787-4ae5-4f80-a2b4-25fd02641543-kube-api-access-6fvcx" (OuterVolumeSpecName: "kube-api-access-6fvcx") pod "923d8787-4ae5-4f80-a2b4-25fd02641543" (UID: "923d8787-4ae5-4f80-a2b4-25fd02641543"). InnerVolumeSpecName "kube-api-access-6fvcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:26:43.865512 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.865476    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rb27\" (UniqueName: \"kubernetes.io/projected/8304e7a6-03e3-4e13-881e-dd2e7089d661-kube-api-access-4rb27\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:26:43.865512 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:43.865504    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fvcx\" (UniqueName: \"kubernetes.io/projected/923d8787-4ae5-4f80-a2b4-25fd02641543-kube-api-access-6fvcx\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:26:44.395852 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.395819    2572 generic.go:356] "Generic (PLEG): container finished" podID="8304e7a6-03e3-4e13-881e-dd2e7089d661" containerID="105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193" exitCode=0
Jul 29 21:26:44.396361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.395861    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-684d965f97-hztjn"
Jul 29 21:26:44.396361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.395893    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684d965f97-hztjn" event={"ID":"8304e7a6-03e3-4e13-881e-dd2e7089d661","Type":"ContainerDied","Data":"105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193"}
Jul 29 21:26:44.396361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.395924    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-684d965f97-hztjn" event={"ID":"8304e7a6-03e3-4e13-881e-dd2e7089d661","Type":"ContainerDied","Data":"5a09edaa4a40734764a36d7c3d79d279bced8bfdb2e4c20b71149e00722524e9"}
Jul 29 21:26:44.396361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.395939    2572 scope.go:117] "RemoveContainer" containerID="105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193"
Jul 29 21:26:44.398408 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.398379    2572 generic.go:356] "Generic (PLEG): container finished" podID="923d8787-4ae5-4f80-a2b4-25fd02641543" containerID="34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb" exitCode=0
Jul 29 21:26:44.398529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.398435    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vwknj"
Jul 29 21:26:44.398529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.398464    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vwknj" event={"ID":"923d8787-4ae5-4f80-a2b4-25fd02641543","Type":"ContainerDied","Data":"34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb"}
Jul 29 21:26:44.398529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.398500    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vwknj" event={"ID":"923d8787-4ae5-4f80-a2b4-25fd02641543","Type":"ContainerDied","Data":"dd71530502277598011ea450ad2a5b95b56039f747a3df68061d145df31da426"}
Jul 29 21:26:44.405011 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.404983    2572 scope.go:117] "RemoveContainer" containerID="105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193"
Jul 29 21:26:44.405355 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:26:44.405329    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193\": container with ID starting with 105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193 not found: ID does not exist" containerID="105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193"
Jul 29 21:26:44.405466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.405366    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193"} err="failed to get container status \"105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193\": rpc error: code = NotFound desc = could not find container \"105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193\": container with ID starting with 105725127e192f5d66123b0e010f8df0192f931f5431b91b72afd3fa07254193 not found: ID does not exist"
Jul 29 21:26:44.405466 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.405389    2572 scope.go:117] "RemoveContainer" containerID="34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb"
Jul 29 21:26:44.415430 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.415409    2572 scope.go:117] "RemoveContainer" containerID="34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb"
Jul 29 21:26:44.415867 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:26:44.415805    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb\": container with ID starting with 34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb not found: ID does not exist" containerID="34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb"
Jul 29 21:26:44.415867 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.415842    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb"} err="failed to get container status \"34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb\": rpc error: code = NotFound desc = could not find container \"34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb\": container with ID starting with 34d32fe1719176288164bc9130fd1888c1d6d5a48f2e2f7624d09352386939bb not found: ID does not exist"
Jul 29 21:26:44.421505 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.421483    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-684d965f97-hztjn"]
Jul 29 21:26:44.425265 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.425243    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-684d965f97-hztjn"]
Jul 29 21:26:44.436371 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.436349    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vwknj"]
Jul 29 21:26:44.441026 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:44.441004    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vwknj"]
Jul 29 21:26:45.305350 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:45.305313    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26dfe913-b90a-4069-92bc-0144a2baa7be" path="/var/lib/kubelet/pods/26dfe913-b90a-4069-92bc-0144a2baa7be/volumes"
Jul 29 21:26:45.305671 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:45.305657    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8304e7a6-03e3-4e13-881e-dd2e7089d661" path="/var/lib/kubelet/pods/8304e7a6-03e3-4e13-881e-dd2e7089d661/volumes"
Jul 29 21:26:45.305939 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:45.305927    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="923d8787-4ae5-4f80-a2b4-25fd02641543" path="/var/lib/kubelet/pods/923d8787-4ae5-4f80-a2b4-25fd02641543/volumes"
Jul 29 21:26:48.241220 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:48.241179    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:48.338242 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:48.338202    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:48.423767 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:48.423736    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:48.543957 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:48.543866    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:51.731475 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:51.731378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:26:51.830603 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:26:51.830567    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:02.068160 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:02.068114    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:02.169105 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:02.169072    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:23.519460 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:23.519421    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:23.621136 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:23.621104    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:25.101083 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:25.101025    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:25.618840 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:25.618802    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:25.732950 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:25.732911    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:25.775585 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:25.775548    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:27:25.832187 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:25.832134    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:37.131280 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:37.131232    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:37.231642 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:37.231604    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:47.785963 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:47.785929    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:27:48.275401 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:27:48.275359    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:28:12.258527 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:12.258488    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:12.947879 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:12.947840    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:28:16.905685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:16.905646    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:17.306686 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.306655    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"]
Jul 29 21:28:17.307013 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307001    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8304e7a6-03e3-4e13-881e-dd2e7089d661" containerName="authorino"
Jul 29 21:28:17.307065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307015    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8304e7a6-03e3-4e13-881e-dd2e7089d661" containerName="authorino"
Jul 29 21:28:17.307065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307026    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="923d8787-4ae5-4f80-a2b4-25fd02641543" containerName="authorino"
Jul 29 21:28:17.307065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307033    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="923d8787-4ae5-4f80-a2b4-25fd02641543" containerName="authorino"
Jul 29 21:28:17.307065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307053    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26dfe913-b90a-4069-92bc-0144a2baa7be" containerName="authorino"
Jul 29 21:28:17.307065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307058    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="26dfe913-b90a-4069-92bc-0144a2baa7be" containerName="authorino"
Jul 29 21:28:17.307302 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307106    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="923d8787-4ae5-4f80-a2b4-25fd02641543" containerName="authorino"
Jul 29 21:28:17.307302 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307118    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="26dfe913-b90a-4069-92bc-0144a2baa7be" containerName="authorino"
Jul 29 21:28:17.307302 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.307125    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8304e7a6-03e3-4e13-881e-dd2e7089d661" containerName="authorino"
Jul 29 21:28:17.310252 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.310231    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.312751 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.312729    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 21:28:17.312989 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.312975    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 21:28:17.313638 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.313598    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8bjkc\""
Jul 29 21:28:17.313854 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.313835    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 29 21:28:17.316893 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.316870    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"]
Jul 29 21:28:17.356129 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.356096    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:17.481633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.481592    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb85914-239b-4006-9495-aaa5f297e1f2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.481820 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.481639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.481820 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.481671    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjmbm\" (UniqueName: \"kubernetes.io/projected/cbb85914-239b-4006-9495-aaa5f297e1f2-kube-api-access-cjmbm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.481820 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.481726    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.481969 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.481815    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.481969 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.481847    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.582596 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.582503    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb85914-239b-4006-9495-aaa5f297e1f2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.582596 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.582546    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.582596 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.582571    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjmbm\" (UniqueName: \"kubernetes.io/projected/cbb85914-239b-4006-9495-aaa5f297e1f2-kube-api-access-cjmbm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.582909 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.582615    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.582909 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.582655    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.582909 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.582674    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.583072 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.583019    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.583072 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.583056    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.583144 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.583089    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.584865 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.584842    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cbb85914-239b-4006-9495-aaa5f297e1f2-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.585143 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.585123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cbb85914-239b-4006-9495-aaa5f297e1f2-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.592064 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.592025    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjmbm\" (UniqueName: \"kubernetes.io/projected/cbb85914-239b-4006-9495-aaa5f297e1f2-kube-api-access-cjmbm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vljm9\" (UID: \"cbb85914-239b-4006-9495-aaa5f297e1f2\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.622033 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.622000    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:17.761780 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.761750    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"]
Jul 29 21:28:17.761899 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:28:17.761833    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbb85914_239b_4006_9495_aaa5f297e1f2.slice/crio-495bf9c72bbd7bf91f8e992909dd0882e626b71a3d1b2cedaf25badfca0433c8 WatchSource:0}: Error finding container 495bf9c72bbd7bf91f8e992909dd0882e626b71a3d1b2cedaf25badfca0433c8: Status 404 returned error can't find the container with id 495bf9c72bbd7bf91f8e992909dd0882e626b71a3d1b2cedaf25badfca0433c8
Jul 29 21:28:17.763279 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.763259    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:28:17.883844 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:17.883757    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9" event={"ID":"cbb85914-239b-4006-9495-aaa5f297e1f2","Type":"ContainerStarted","Data":"495bf9c72bbd7bf91f8e992909dd0882e626b71a3d1b2cedaf25badfca0433c8"}
Jul 29 21:28:18.441478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:18.441442    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:28:18.891225 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:18.891192    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9" event={"ID":"cbb85914-239b-4006-9495-aaa5f297e1f2","Type":"ContainerStarted","Data":"74b21a1192df7c392ec73b658da1a0028cacb2b19240e136b077ceb587997b34"}
Jul 29 21:28:18.911685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:18.911619    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9" podStartSLOduration=0.901749515 podStartE2EDuration="1.91159647s" totalImagesPullingTime="1.009846955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:28:18.91101555 +0000 UTC m=+710.127793015" watchObservedRunningTime="2026-07-29 21:28:18.91159647 +0000 UTC m=+710.128373935"
Jul 29 21:28:27.622760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:27.622649    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:27.622760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:27.622755    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:27.635583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:27.635550    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:27.953080 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:27.953052    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vljm9"
Jul 29 21:28:32.052340 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.052301    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"]
Jul 29 21:28:32.056119 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.056097    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.059676 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.059655    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 21:28:32.074255 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.074226    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"]
Jul 29 21:28:32.102743 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.102710    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.102916 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.102754    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.102916 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.102786    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.102916 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.102875    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4hlg\" (UniqueName: \"kubernetes.io/projected/a7e1eb19-ab90-41f8-82e3-9ea908290774-kube-api-access-p4hlg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.103052 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.102923    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7e1eb19-ab90-41f8-82e3-9ea908290774-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.103052 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.102995    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.143412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.143369    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:32.204042 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204002    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204042 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204048    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204082    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204113    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204159    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4hlg\" (UniqueName: \"kubernetes.io/projected/a7e1eb19-ab90-41f8-82e3-9ea908290774-kube-api-access-p4hlg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204209    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7e1eb19-ab90-41f8-82e3-9ea908290774-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204542 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204470    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204542 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204520    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.204626 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.204554    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.206510 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.206490    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a7e1eb19-ab90-41f8-82e3-9ea908290774-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.206672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.206654    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7e1eb19-ab90-41f8-82e3-9ea908290774-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.213750 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.213723    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4hlg\" (UniqueName: \"kubernetes.io/projected/a7e1eb19-ab90-41f8-82e3-9ea908290774-kube-api-access-p4hlg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p\" (UID: \"a7e1eb19-ab90-41f8-82e3-9ea908290774\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.366314 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.366214    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:32.515022 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.514993    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"]
Jul 29 21:28:32.517090 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:28:32.517057    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e1eb19_ab90_41f8_82e3_9ea908290774.slice/crio-c4b59cf07178000e55f38dd47268d77fac6fdb676d68aef266cf5f06d846cc63 WatchSource:0}: Error finding container c4b59cf07178000e55f38dd47268d77fac6fdb676d68aef266cf5f06d846cc63: Status 404 returned error can't find the container with id c4b59cf07178000e55f38dd47268d77fac6fdb676d68aef266cf5f06d846cc63
Jul 29 21:28:32.972330 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.972292    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p" event={"ID":"a7e1eb19-ab90-41f8-82e3-9ea908290774","Type":"ContainerStarted","Data":"0ef5cee32ad7941156e9a1379e3dc45627ae8149eed1bb0dc1b91da8925eb424"}
Jul 29 21:28:32.972630 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.972348    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p" event={"ID":"a7e1eb19-ab90-41f8-82e3-9ea908290774","Type":"ContainerStarted","Data":"c4b59cf07178000e55f38dd47268d77fac6fdb676d68aef266cf5f06d846cc63"}
Jul 29 21:28:32.996436 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:32.996322    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p" podStartSLOduration=0.780349555 podStartE2EDuration="996.304623ms" totalImagesPullingTime="215.955068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:28:32.995235979 +0000 UTC m=+724.212013452" watchObservedRunningTime="2026-07-29 21:28:32.996304623 +0000 UTC m=+724.213082082"
Jul 29 21:28:33.645498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:33.645462    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:28:42.366555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:42.366519    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:42.366966 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:42.366582    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:42.378903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:42.378880    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:43.035120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:43.035088    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p"
Jul 29 21:28:47.458983 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:47.458947    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:50.125052 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:50.125013    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:28:57.001108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.001066    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"]
Jul 29 21:28:57.004728 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.004706    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.007213 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.007191    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 29 21:28:57.015536 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.015509    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"]
Jul 29 21:28:57.131141 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.131093    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:28:57.137112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.137086    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/987ea528-3f26-40ea-9851-794a5d9aa16b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.137258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.137122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.137258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.137144    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.137258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.137182    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtgjv\" (UniqueName: \"kubernetes.io/projected/987ea528-3f26-40ea-9851-794a5d9aa16b-kube-api-access-rtgjv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.137376 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.137283    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.137376 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.137320    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.238528 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.238485    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.238736 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.238542    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.238736 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.238583    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/987ea528-3f26-40ea-9851-794a5d9aa16b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.238736 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.238602    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.238736 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.238617    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.238736 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.238635    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtgjv\" (UniqueName: \"kubernetes.io/projected/987ea528-3f26-40ea-9851-794a5d9aa16b-kube-api-access-rtgjv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.239056 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.239030    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.239123 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.239066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.239218 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.239160    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.240821 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.240800    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/987ea528-3f26-40ea-9851-794a5d9aa16b-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.241154 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.241136    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/987ea528-3f26-40ea-9851-794a5d9aa16b-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.247221 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.247154    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtgjv\" (UniqueName: \"kubernetes.io/projected/987ea528-3f26-40ea-9851-794a5d9aa16b-kube-api-access-rtgjv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-7p25z\" (UID: \"987ea528-3f26-40ea-9851-794a5d9aa16b\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.315585 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.315500    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:28:57.456530 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:57.456505    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"]
Jul 29 21:28:57.458508 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:28:57.458478    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod987ea528_3f26_40ea_9851_794a5d9aa16b.slice/crio-796adcc061f0790dd5b2e73993b99e3996a34e3d8d81bed8c9053b5a32f93d22 WatchSource:0}: Error finding container 796adcc061f0790dd5b2e73993b99e3996a34e3d8d81bed8c9053b5a32f93d22: Status 404 returned error can't find the container with id 796adcc061f0790dd5b2e73993b99e3996a34e3d8d81bed8c9053b5a32f93d22
Jul 29 21:28:58.107743 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:58.107712    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z" event={"ID":"987ea528-3f26-40ea-9851-794a5d9aa16b","Type":"ContainerStarted","Data":"4b8d5984f913b2ba2e1fa163b4f5cd68f1f3148f7f0d8c5eb08be4fff2baa5a0"}
Jul 29 21:28:58.108163 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:58.107754    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z" event={"ID":"987ea528-3f26-40ea-9851-794a5d9aa16b","Type":"ContainerStarted","Data":"796adcc061f0790dd5b2e73993b99e3996a34e3d8d81bed8c9053b5a32f93d22"}
Jul 29 21:28:58.136925 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:58.136879    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z" podStartSLOduration=1.949470617 podStartE2EDuration="2.136863218s" totalImagesPullingTime="187.392601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:28:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:28:58.13670914 +0000 UTC m=+749.353486631" watchObservedRunningTime="2026-07-29 21:28:58.136863218 +0000 UTC m=+749.353640709"
Jul 29 21:28:59.834615 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:28:59.834567    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:29:07.315964 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:07.315925    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:29:07.316487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:07.316281    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:29:07.329135 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:07.329105    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:29:08.178040 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:08.178009    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-7p25z"
Jul 29 21:29:30.751387 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:30.751345    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:30.813673 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:30.813631    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:36.667676 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:36.667636    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:55.931640 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:55.931549    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57b9dd47c6-qsw72"]
Jul 29 21:29:55.935270 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:55.935245    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:55.942869 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:55.942845    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57b9dd47c6-qsw72"]
Jul 29 21:29:56.087756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.087720    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f78fb90c-a02b-488c-be41-142a87e9fb6d-tls-cert\") pod \"authorino-57b9dd47c6-qsw72\" (UID: \"f78fb90c-a02b-488c-be41-142a87e9fb6d\") " pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.087947 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.087773    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzmwl\" (UniqueName: \"kubernetes.io/projected/f78fb90c-a02b-488c-be41-142a87e9fb6d-kube-api-access-lzmwl\") pod \"authorino-57b9dd47c6-qsw72\" (UID: \"f78fb90c-a02b-488c-be41-142a87e9fb6d\") " pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.189259 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.189154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f78fb90c-a02b-488c-be41-142a87e9fb6d-tls-cert\") pod \"authorino-57b9dd47c6-qsw72\" (UID: \"f78fb90c-a02b-488c-be41-142a87e9fb6d\") " pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.189259 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.189250    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzmwl\" (UniqueName: \"kubernetes.io/projected/f78fb90c-a02b-488c-be41-142a87e9fb6d-kube-api-access-lzmwl\") pod \"authorino-57b9dd47c6-qsw72\" (UID: \"f78fb90c-a02b-488c-be41-142a87e9fb6d\") " pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.191886 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.191857    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f78fb90c-a02b-488c-be41-142a87e9fb6d-tls-cert\") pod \"authorino-57b9dd47c6-qsw72\" (UID: \"f78fb90c-a02b-488c-be41-142a87e9fb6d\") " pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.197085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.197059    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzmwl\" (UniqueName: \"kubernetes.io/projected/f78fb90c-a02b-488c-be41-142a87e9fb6d-kube-api-access-lzmwl\") pod \"authorino-57b9dd47c6-qsw72\" (UID: \"f78fb90c-a02b-488c-be41-142a87e9fb6d\") " pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.245101 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.245069    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57b9dd47c6-qsw72"
Jul 29 21:29:56.382659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.382626    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57b9dd47c6-qsw72"]
Jul 29 21:29:56.384341 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:29:56.384299    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf78fb90c_a02b_488c_be41_142a87e9fb6d.slice/crio-cd647cdc8697ba205ebbe58b5dfcf101027c378e31388a6ed63e64eced27bad8 WatchSource:0}: Error finding container cd647cdc8697ba205ebbe58b5dfcf101027c378e31388a6ed63e64eced27bad8: Status 404 returned error can't find the container with id cd647cdc8697ba205ebbe58b5dfcf101027c378e31388a6ed63e64eced27bad8
Jul 29 21:29:56.437686 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:56.437646    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57b9dd47c6-qsw72" event={"ID":"f78fb90c-a02b-488c-be41-142a87e9fb6d","Type":"ContainerStarted","Data":"cd647cdc8697ba205ebbe58b5dfcf101027c378e31388a6ed63e64eced27bad8"}
Jul 29 21:29:57.448550 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:57.448502    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57b9dd47c6-qsw72" event={"ID":"f78fb90c-a02b-488c-be41-142a87e9fb6d","Type":"ContainerStarted","Data":"95ef8f1d033f87606cf1be117b9d533fb9a9ce186e441714e196f6c18cfebd56"}
Jul 29 21:29:57.471258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:57.471193    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57b9dd47c6-qsw72" podStartSLOduration=2.116238967 podStartE2EDuration="2.471153379s" totalImagesPullingTime="354.914412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:29:57.470668416 +0000 UTC m=+808.687445879" watchObservedRunningTime="2026-07-29 21:29:57.471153379 +0000 UTC m=+808.687930842"
Jul 29 21:29:57.549758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:57.549724    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c77c8474f-bbvxg"]
Jul 29 21:29:57.550002 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:57.549967    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5c77c8474f-bbvxg" podUID="0a9c1fe9-e48f-47b5-b355-2704f25f46d8" containerName="authorino" containerID="cri-o://bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d" gracePeriod=30
Jul 29 21:29:57.776258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:57.776074    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:57.821162 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:57.821133    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:29:58.008428 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.008349    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqj7w\" (UniqueName: \"kubernetes.io/projected/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-kube-api-access-vqj7w\") pod \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") "
Jul 29 21:29:58.008578 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.008478    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-tls-cert\") pod \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\" (UID: \"0a9c1fe9-e48f-47b5-b355-2704f25f46d8\") "
Jul 29 21:29:58.010664 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.010629    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-kube-api-access-vqj7w" (OuterVolumeSpecName: "kube-api-access-vqj7w") pod "0a9c1fe9-e48f-47b5-b355-2704f25f46d8" (UID: "0a9c1fe9-e48f-47b5-b355-2704f25f46d8"). InnerVolumeSpecName "kube-api-access-vqj7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:29:58.020800 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.020766    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0a9c1fe9-e48f-47b5-b355-2704f25f46d8" (UID: "0a9c1fe9-e48f-47b5-b355-2704f25f46d8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:29:58.089746 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.089699    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:58.110125 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.110094    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqj7w\" (UniqueName: \"kubernetes.io/projected/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-kube-api-access-vqj7w\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:29:58.110125 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.110125    2572 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0a9c1fe9-e48f-47b5-b355-2704f25f46d8-tls-cert\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:29:58.180505 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.180447    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:58.457860 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.457825    2572 generic.go:356] "Generic (PLEG): container finished" podID="0a9c1fe9-e48f-47b5-b355-2704f25f46d8" containerID="bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d" exitCode=0
Jul 29 21:29:58.458363 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.457885    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5c77c8474f-bbvxg"
Jul 29 21:29:58.458363 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.457915    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77c8474f-bbvxg" event={"ID":"0a9c1fe9-e48f-47b5-b355-2704f25f46d8","Type":"ContainerDied","Data":"bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d"}
Jul 29 21:29:58.458363 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.457959    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5c77c8474f-bbvxg" event={"ID":"0a9c1fe9-e48f-47b5-b355-2704f25f46d8","Type":"ContainerDied","Data":"4e188b26ffc5043265ded901801356aa2566fa92079f658a7f3e19e0702b8b9d"}
Jul 29 21:29:58.458363 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.457981    2572 scope.go:117] "RemoveContainer" containerID="bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d"
Jul 29 21:29:58.466979 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.466780    2572 scope.go:117] "RemoveContainer" containerID="bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d"
Jul 29 21:29:58.467256 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:29:58.467227    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d\": container with ID starting with bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d not found: ID does not exist" containerID="bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d"
Jul 29 21:29:58.467363 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.467265    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d"} err="failed to get container status \"bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d\": rpc error: code = NotFound desc = could not find container \"bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d\": container with ID starting with bfdbee4c30d6c4996fedd1ad34897a81f84b7b4bbb9ff7a748533ae24cd9237d not found: ID does not exist"
Jul 29 21:29:58.487693 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.487654    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5c77c8474f-bbvxg"]
Jul 29 21:29:58.493850 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:58.493825    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5c77c8474f-bbvxg"]
Jul 29 21:29:59.227940 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:59.227900    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:59.305769 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:59.305733    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a9c1fe9-e48f-47b5-b355-2704f25f46d8" path="/var/lib/kubelet/pods/0a9c1fe9-e48f-47b5-b355-2704f25f46d8/volumes"
Jul 29 21:29:59.334491 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:59.334457    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:29:59.748765 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:29:59.748731    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:01.019649 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:30:01.019608    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:01.099678 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:30:01.099639    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:30:26.931077 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:30:26.931034    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:30:37.634867 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:30:37.634832    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:30:48.923537 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:30:48.923500    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:32:04.006747 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:32:04.006715    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:32:05.785892 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:32:05.785859    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:32:30.436185 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:32:30.436124    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:32:42.821282 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:32:42.821245    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:33:20.618053 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:20.618016    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:20.717997 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:20.717959    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:20.829911 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:20.829877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:26.825922 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:26.825889    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:26.906335 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:26.906301    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:27.001051 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:27.001014    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:27.190977 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:27.190945    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:32.983623 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:32.983587    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:33.057599 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:33.057565    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:48.831609 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:48.831570    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:33:54.107161 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:54.107118    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:33:56.528116 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:33:56.528080    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:34:06.371984 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:06.371945    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:34:11.837833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:11.837785    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:34:21.125055 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:21.125013    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:34:30.122783 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:30.122701    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:34:41.825275 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:41.825238    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:34:49.743851 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:49.743815    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:34:59.735316 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:34:59.735275    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:35:22.538207 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:35:22.538153    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:35:36.044036 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:35:36.044002    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:35:57.123513 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:35:57.123434    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:36:07.918975 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:36:07.918934    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:36:25.327665 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:36:25.327623    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:36:44.122925 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:36:44.122882    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:36:51.530243 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:36:51.530208    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:37:07.034863 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:37:07.034823    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:37:24.164625 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:37:24.164541    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:37:32.883219 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:37:32.883165    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:37:42.771719 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:37:42.771678    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:38:02.831319 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:38:02.831277    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:38:14.919493 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:38:14.919460    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:38:22.527004 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:38:22.526970    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:38:48.779426 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:38:48.779393    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:38:48.844410 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:38:48.844345    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:39:12.853078 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:12.852990    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"]
Jul 29 21:39:12.853655 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:12.853247    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq" podUID="c0327482-5999-42f7-a015-9a30d10f6ab9" containerName="manager" containerID="cri-o://9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b" gracePeriod=10
Jul 29 21:39:13.116320 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.116246    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:39:13.224144 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.224111    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lz5p\" (UniqueName: \"kubernetes.io/projected/c0327482-5999-42f7-a015-9a30d10f6ab9-kube-api-access-5lz5p\") pod \"c0327482-5999-42f7-a015-9a30d10f6ab9\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") "
Jul 29 21:39:13.224372 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.224202    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0327482-5999-42f7-a015-9a30d10f6ab9-extensions-socket-volume\") pod \"c0327482-5999-42f7-a015-9a30d10f6ab9\" (UID: \"c0327482-5999-42f7-a015-9a30d10f6ab9\") "
Jul 29 21:39:13.224561 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.224530    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0327482-5999-42f7-a015-9a30d10f6ab9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c0327482-5999-42f7-a015-9a30d10f6ab9" (UID: "c0327482-5999-42f7-a015-9a30d10f6ab9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:39:13.226230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.226209    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0327482-5999-42f7-a015-9a30d10f6ab9-kube-api-access-5lz5p" (OuterVolumeSpecName: "kube-api-access-5lz5p") pod "c0327482-5999-42f7-a015-9a30d10f6ab9" (UID: "c0327482-5999-42f7-a015-9a30d10f6ab9"). InnerVolumeSpecName "kube-api-access-5lz5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:39:13.325629 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.325587    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lz5p\" (UniqueName: \"kubernetes.io/projected/c0327482-5999-42f7-a015-9a30d10f6ab9-kube-api-access-5lz5p\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:39:13.325629 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.325617    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0327482-5999-42f7-a015-9a30d10f6ab9-extensions-socket-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:39:13.500532 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.500493    2572 generic.go:356] "Generic (PLEG): container finished" podID="c0327482-5999-42f7-a015-9a30d10f6ab9" containerID="9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b" exitCode=0
Jul 29 21:39:13.500708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.500543    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"
Jul 29 21:39:13.500708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.500564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq" event={"ID":"c0327482-5999-42f7-a015-9a30d10f6ab9","Type":"ContainerDied","Data":"9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b"}
Jul 29 21:39:13.500708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.500618    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq" event={"ID":"c0327482-5999-42f7-a015-9a30d10f6ab9","Type":"ContainerDied","Data":"cf5e729ada9760fd316591f65dccc70dd5af80f97e7da7951c050642c9e2a096"}
Jul 29 21:39:13.500708 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.500649    2572 scope.go:117] "RemoveContainer" containerID="9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b"
Jul 29 21:39:13.510636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.510617    2572 scope.go:117] "RemoveContainer" containerID="9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b"
Jul 29 21:39:13.510914 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:39:13.510888    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b\": container with ID starting with 9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b not found: ID does not exist" containerID="9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b"
Jul 29 21:39:13.510968 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.510924    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b"} err="failed to get container status \"9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b\": rpc error: code = NotFound desc = could not find container \"9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b\": container with ID starting with 9727c39a66a7fc8280577bbe50abcdd758bcd31ae4bf986646b2e0b51934198b not found: ID does not exist"
Jul 29 21:39:13.519921 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.519894    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"]
Jul 29 21:39:13.531138 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:13.531115    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g7mqq"]
Jul 29 21:39:15.305187 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:39:15.305134    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0327482-5999-42f7-a015-9a30d10f6ab9" path="/var/lib/kubelet/pods/c0327482-5999-42f7-a015-9a30d10f6ab9/volumes"
Jul 29 21:40:13.590638 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.590596    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"]
Jul 29 21:40:13.591139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.591015    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a9c1fe9-e48f-47b5-b355-2704f25f46d8" containerName="authorino"
Jul 29 21:40:13.591139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.591030    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a9c1fe9-e48f-47b5-b355-2704f25f46d8" containerName="authorino"
Jul 29 21:40:13.591139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.591060    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0327482-5999-42f7-a015-9a30d10f6ab9" containerName="manager"
Jul 29 21:40:13.591139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.591065    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0327482-5999-42f7-a015-9a30d10f6ab9" containerName="manager"
Jul 29 21:40:13.591139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.591127    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0327482-5999-42f7-a015-9a30d10f6ab9" containerName="manager"
Jul 29 21:40:13.591139 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.591137    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a9c1fe9-e48f-47b5-b355-2704f25f46d8" containerName="authorino"
Jul 29 21:40:13.594212 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.594193    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.596691 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.596664    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nm827\""
Jul 29 21:40:13.605404 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.605379    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"]
Jul 29 21:40:13.654928 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.654899    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m57v6\" (UniqueName: \"kubernetes.io/projected/07d33eee-10b0-4d5a-bb14-6e6736748bf1-kube-api-access-m57v6\") pod \"kuadrant-operator-controller-manager-86878c769f-mjp56\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.655097 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.654953    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07d33eee-10b0-4d5a-bb14-6e6736748bf1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mjp56\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.755623 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.755577    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07d33eee-10b0-4d5a-bb14-6e6736748bf1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mjp56\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.755796 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.755715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m57v6\" (UniqueName: \"kubernetes.io/projected/07d33eee-10b0-4d5a-bb14-6e6736748bf1-kube-api-access-m57v6\") pod \"kuadrant-operator-controller-manager-86878c769f-mjp56\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.756007 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.755989    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07d33eee-10b0-4d5a-bb14-6e6736748bf1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mjp56\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.765506 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.765474    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m57v6\" (UniqueName: \"kubernetes.io/projected/07d33eee-10b0-4d5a-bb14-6e6736748bf1-kube-api-access-m57v6\") pod \"kuadrant-operator-controller-manager-86878c769f-mjp56\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:13.906291 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:13.906191    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:14.053519 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:14.053492    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"]
Jul 29 21:40:14.055068 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:40:14.055041    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d33eee_10b0_4d5a_bb14_6e6736748bf1.slice/crio-6878a886c6858317a093e459a36e5156cfda9ea7609892062f89d58e3a581b9a WatchSource:0}: Error finding container 6878a886c6858317a093e459a36e5156cfda9ea7609892062f89d58e3a581b9a: Status 404 returned error can't find the container with id 6878a886c6858317a093e459a36e5156cfda9ea7609892062f89d58e3a581b9a
Jul 29 21:40:14.830026 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:14.829996    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56" event={"ID":"07d33eee-10b0-4d5a-bb14-6e6736748bf1","Type":"ContainerStarted","Data":"676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca"}
Jul 29 21:40:14.830507 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:14.830043    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56" event={"ID":"07d33eee-10b0-4d5a-bb14-6e6736748bf1","Type":"ContainerStarted","Data":"6878a886c6858317a093e459a36e5156cfda9ea7609892062f89d58e3a581b9a"}
Jul 29 21:40:14.830507 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:14.830074    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:14.865881 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:14.865823    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56" podStartSLOduration=1.8658024850000001 podStartE2EDuration="1.865802485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:40:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:40:14.862799689 +0000 UTC m=+1426.079577151" watchObservedRunningTime="2026-07-29 21:40:14.865802485 +0000 UTC m=+1426.082579948"
Jul 29 21:40:25.837974 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:25.837939    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:40:36.023161 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:36.023126    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:40:43.743195 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:40:43.743142    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:41:07.572411 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:41:07.572378    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:41:47.780226 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:41:47.780187    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:41:53.455136 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:41:53.455034    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:41:57.175939 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:41:57.175903    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:42:09.996060 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:42:09.996023    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:42:20.941675 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:42:20.941623    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:42:41.282728 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:42:41.282691    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:42:51.637006 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:42:51.636965    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:43:02.598757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:02.598719    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:43:12.633310 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:12.633268    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:43:30.926054 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:30.925966    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:43:40.144857 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.144821    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:43:40.235989 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.235949    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"]
Jul 29 21:43:40.239544 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.239522    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.242309 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.242287    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-2b453af4-kserve-self-signed-certs\""
Jul 29 21:43:40.266563 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.266534    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"]
Jul 29 21:43:40.322863 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.322823    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:40.363463 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.363429    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-home\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.363655 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.363502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-model-cache\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.363655 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.363527    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tls-certs\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.363655 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.363547    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-dshm\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.363655 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.363571    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqqcd\" (UniqueName: \"kubernetes.io/projected/c0bb417c-a570-4cc1-9d37-e1655e479cdf-kube-api-access-jqqcd\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.363903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.363656    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tmp-dir\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.464908 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.464424    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-model-cache\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.464927    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tls-certs\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.464964    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-dshm\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.465007    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqqcd\" (UniqueName: \"kubernetes.io/projected/c0bb417c-a570-4cc1-9d37-e1655e479cdf-kube-api-access-jqqcd\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.465066    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tmp-dir\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.465133    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-home\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465470 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.464859    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-model-cache\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465914 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.465810    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-home\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.465914 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.465862    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tmp-dir\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.467851 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.467827    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-dshm\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.468729 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.468709    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tls-certs\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.474179 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.474142    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqqcd\" (UniqueName: \"kubernetes.io/projected/c0bb417c-a570-4cc1-9d37-e1655e479cdf-kube-api-access-jqqcd\") pod \"e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") " pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.552108 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.552072    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:40.696852 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.696827    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"]
Jul 29 21:43:40.699564 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:43:40.699525    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0bb417c_a570_4cc1_9d37_e1655e479cdf.slice/crio-75f2515a7a743d12c1436e82e53549b9f3518b3accc75ba9d4b05a9f3f7d0eab WatchSource:0}: Error finding container 75f2515a7a743d12c1436e82e53549b9f3518b3accc75ba9d4b05a9f3f7d0eab: Status 404 returned error can't find the container with id 75f2515a7a743d12c1436e82e53549b9f3518b3accc75ba9d4b05a9f3f7d0eab
Jul 29 21:43:40.963824 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.963786    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp" event={"ID":"c0bb417c-a570-4cc1-9d37-e1655e479cdf","Type":"ContainerStarted","Data":"2e50d0b1c3dc9281f1704aed66f225f79b6d7eba24c7e2983ea90d1fbb9c4b4f"}
Jul 29 21:43:40.963824 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.963831    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp" event={"ID":"c0bb417c-a570-4cc1-9d37-e1655e479cdf","Type":"ContainerStarted","Data":"75f2515a7a743d12c1436e82e53549b9f3518b3accc75ba9d4b05a9f3f7d0eab"}
Jul 29 21:43:40.982218 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:40.981714    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp" podStartSLOduration=0.981692621 podStartE2EDuration="981.692621ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:43:40.981329507 +0000 UTC m=+1632.198106971" watchObservedRunningTime="2026-07-29 21:43:40.981692621 +0000 UTC m=+1632.198470084"
Jul 29 21:43:50.552965 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:50.552877    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:50.553420 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:50.553137    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:50.565792 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:50.565763    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:51.030232 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:51.030201    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:43:52.337848 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:52.337809    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:43:56.805165 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.805129    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"]
Jul 29 21:43:56.808987 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.808968    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:56.811685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.811666    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-2b453af4-kserve-self-signed-certs\""
Jul 29 21:43:56.825649 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.825615    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"]
Jul 29 21:43:56.916982 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.916931    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:43:56.927185 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.927146    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tmp-dir\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:56.927297 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.927240    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-model-cache\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:56.927297 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.927275    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tls-certs\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:56.927391 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.927371    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-home\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:56.927440 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.927430    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdhb8\" (UniqueName: \"kubernetes.io/projected/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-kube-api-access-rdhb8\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:56.927521 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:56.927499    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-dshm\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028317    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tmp-dir\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028386    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-model-cache\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028412    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tls-certs\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028474    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-home\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028533    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdhb8\" (UniqueName: \"kubernetes.io/projected/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-kube-api-access-rdhb8\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028564    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-dshm\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028847    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tmp-dir\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028872    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-model-cache\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.028937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.028916    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-home\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.030838 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.030805    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-dshm\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.030967 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.030881    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tls-certs\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.036391 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.036363    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdhb8\" (UniqueName: \"kubernetes.io/projected/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-kube-api-access-rdhb8\") pod \"e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") " pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.119375 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.119262    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:43:57.267586 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:57.267458    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"]
Jul 29 21:43:57.270524 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:43:57.270485    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bc8435b_7acd_492d_80d8_48cd2a6b7cf8.slice/crio-67926939bfda9595b56389dccd36b9fabb1228566c8eb0e516940b772686a639 WatchSource:0}: Error finding container 67926939bfda9595b56389dccd36b9fabb1228566c8eb0e516940b772686a639: Status 404 returned error can't find the container with id 67926939bfda9595b56389dccd36b9fabb1228566c8eb0e516940b772686a639
Jul 29 21:43:58.061617 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:58.061577    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd" event={"ID":"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8","Type":"ContainerStarted","Data":"73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800"}
Jul 29 21:43:58.061996 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:58.061624    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd" event={"ID":"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8","Type":"ContainerStarted","Data":"67926939bfda9595b56389dccd36b9fabb1228566c8eb0e516940b772686a639"}
Jul 29 21:43:58.087842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:43:58.087793    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd" podStartSLOduration=2.087777182 podStartE2EDuration="2.087777182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:43:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:43:58.086266114 +0000 UTC m=+1649.303043578" watchObservedRunningTime="2026-07-29 21:43:58.087777182 +0000 UTC m=+1649.304554644"
Jul 29 21:44:00.946358 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:00.946319    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:44:07.119896 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:07.119863    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:44:07.120412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:07.119925    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:44:07.132541 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:07.132515    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:44:08.132099 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:08.132060    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:44:28.157194 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:28.157148    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"]
Jul 29 21:44:28.182514 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:28.182479    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:28.926155 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:28.926119    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"]
Jul 29 21:44:28.948814 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:28.948783    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:44:32.136935 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:32.136895    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:44:43.177065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.177014    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp" podUID="c0bb417c-a570-4cc1-9d37-e1655e479cdf" containerName="main" containerID="cri-o://2e50d0b1c3dc9281f1704aed66f225f79b6d7eba24c7e2983ea90d1fbb9c4b4f" gracePeriod=45
Jul 29 21:44:43.316478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.316438    2572 generic.go:356] "Generic (PLEG): container finished" podID="c0bb417c-a570-4cc1-9d37-e1655e479cdf" containerID="2e50d0b1c3dc9281f1704aed66f225f79b6d7eba24c7e2983ea90d1fbb9c4b4f" exitCode=0
Jul 29 21:44:43.316639 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.316508    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp" event={"ID":"c0bb417c-a570-4cc1-9d37-e1655e479cdf","Type":"ContainerDied","Data":"2e50d0b1c3dc9281f1704aed66f225f79b6d7eba24c7e2983ea90d1fbb9c4b4f"}
Jul 29 21:44:43.456120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.456085    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:44:43.562894 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.562862    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-dshm\") pod \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") "
Jul 29 21:44:43.563085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.562914    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tmp-dir\") pod \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") "
Jul 29 21:44:43.563085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.562972    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-home\") pod \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") "
Jul 29 21:44:43.563085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.562992    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-model-cache\") pod \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") "
Jul 29 21:44:43.563085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.563021    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tls-certs\") pod \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") "
Jul 29 21:44:43.563085 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.563046    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqqcd\" (UniqueName: \"kubernetes.io/projected/c0bb417c-a570-4cc1-9d37-e1655e479cdf-kube-api-access-jqqcd\") pod \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\" (UID: \"c0bb417c-a570-4cc1-9d37-e1655e479cdf\") "
Jul 29 21:44:43.563695 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.563569    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-home" (OuterVolumeSpecName: "home") pod "c0bb417c-a570-4cc1-9d37-e1655e479cdf" (UID: "c0bb417c-a570-4cc1-9d37-e1655e479cdf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:43.563695 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.563600    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c0bb417c-a570-4cc1-9d37-e1655e479cdf" (UID: "c0bb417c-a570-4cc1-9d37-e1655e479cdf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:43.563695 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.563688    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-model-cache" (OuterVolumeSpecName: "model-cache") pod "c0bb417c-a570-4cc1-9d37-e1655e479cdf" (UID: "c0bb417c-a570-4cc1-9d37-e1655e479cdf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:43.565773 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.565750    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-dshm" (OuterVolumeSpecName: "dshm") pod "c0bb417c-a570-4cc1-9d37-e1655e479cdf" (UID: "c0bb417c-a570-4cc1-9d37-e1655e479cdf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:43.565922 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.565847    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c0bb417c-a570-4cc1-9d37-e1655e479cdf" (UID: "c0bb417c-a570-4cc1-9d37-e1655e479cdf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:44:43.566135 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.566106    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0bb417c-a570-4cc1-9d37-e1655e479cdf-kube-api-access-jqqcd" (OuterVolumeSpecName: "kube-api-access-jqqcd") pod "c0bb417c-a570-4cc1-9d37-e1655e479cdf" (UID: "c0bb417c-a570-4cc1-9d37-e1655e479cdf"). InnerVolumeSpecName "kube-api-access-jqqcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:44:43.665114 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.665068    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:43.665114 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.665111    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:43.665365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.665125    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:43.665365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.665140    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqqcd\" (UniqueName: \"kubernetes.io/projected/c0bb417c-a570-4cc1-9d37-e1655e479cdf-kube-api-access-jqqcd\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:43.665365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.665152    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:43.665365 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.665165    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0bb417c-a570-4cc1-9d37-e1655e479cdf-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:43.945684 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:43.945603    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd" podUID="2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" containerName="main" containerID="cri-o://73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800" gracePeriod=45
Jul 29 21:44:44.186359 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.186326    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:44:44.323769 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.323681    2572 generic.go:356] "Generic (PLEG): container finished" podID="2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" containerID="73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800" exitCode=0
Jul 29 21:44:44.323769 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.323717    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"
Jul 29 21:44:44.323985 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.323766    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd" event={"ID":"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8","Type":"ContainerDied","Data":"73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800"}
Jul 29 21:44:44.323985 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.323811    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd" event={"ID":"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8","Type":"ContainerDied","Data":"67926939bfda9595b56389dccd36b9fabb1228566c8eb0e516940b772686a639"}
Jul 29 21:44:44.323985 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.323829    2572 scope.go:117] "RemoveContainer" containerID="73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800"
Jul 29 21:44:44.325132 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.325116    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"
Jul 29 21:44:44.325132 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.325121    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp" event={"ID":"c0bb417c-a570-4cc1-9d37-e1655e479cdf","Type":"ContainerDied","Data":"75f2515a7a743d12c1436e82e53549b9f3518b3accc75ba9d4b05a9f3f7d0eab"}
Jul 29 21:44:44.335702 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.333959    2572 scope.go:117] "RemoveContainer" containerID="73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800"
Jul 29 21:44:44.335957 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:44:44.335762    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800\": container with ID starting with 73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800 not found: ID does not exist" containerID="73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800"
Jul 29 21:44:44.335957 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.335793    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800"} err="failed to get container status \"73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800\": rpc error: code = NotFound desc = could not find container \"73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800\": container with ID starting with 73d4af0034ecd4d9dcb540b6d63ebf603ffcee5672627e1b8efc7c8bc2809800 not found: ID does not exist"
Jul 29 21:44:44.335957 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.335818    2572 scope.go:117] "RemoveContainer" containerID="2e50d0b1c3dc9281f1704aed66f225f79b6d7eba24c7e2983ea90d1fbb9c4b4f"
Jul 29 21:44:44.354776 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.354750    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"]
Jul 29 21:44:44.359438 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.359415    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-2b453af4-kserve-85b6dd676c-m8jzp"]
Jul 29 21:44:44.372453 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372421    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-model-cache\") pod \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") "
Jul 29 21:44:44.372598 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372475    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tls-certs\") pod \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") "
Jul 29 21:44:44.372598 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372495    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tmp-dir\") pod \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") "
Jul 29 21:44:44.372598 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372573    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdhb8\" (UniqueName: \"kubernetes.io/projected/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-kube-api-access-rdhb8\") pod \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") "
Jul 29 21:44:44.372598 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372599    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-dshm\") pod \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") "
Jul 29 21:44:44.372829 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372673    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-home\") pod \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\" (UID: \"2bc8435b-7acd-492d-80d8-48cd2a6b7cf8\") "
Jul 29 21:44:44.372829 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372699    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-model-cache" (OuterVolumeSpecName: "model-cache") pod "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" (UID: "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:44.372939 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372840    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" (UID: "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:44.373000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372965    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-home" (OuterVolumeSpecName: "home") pod "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" (UID: "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:44.373000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372982    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:44.373000 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.372999    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:44.374827 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.374798    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-dshm" (OuterVolumeSpecName: "dshm") pod "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" (UID: "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:44:44.374827 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.374810    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" (UID: "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:44:44.374975 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.374809    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-kube-api-access-rdhb8" (OuterVolumeSpecName: "kube-api-access-rdhb8") pod "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" (UID: "2bc8435b-7acd-492d-80d8-48cd2a6b7cf8"). InnerVolumeSpecName "kube-api-access-rdhb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:44:44.473612 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.473566    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdhb8\" (UniqueName: \"kubernetes.io/projected/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-kube-api-access-rdhb8\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:44.473612 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.473608    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:44.473833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.473625    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:44.473833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.473643    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:44:44.646573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.646539    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"]
Jul 29 21:44:44.653301 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:44.653279    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-2b453af4-kserve-5fb7698895-xwvnd"]
Jul 29 21:44:45.304445 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:45.304412    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" path="/var/lib/kubelet/pods/2bc8435b-7acd-492d-80d8-48cd2a6b7cf8/volumes"
Jul 29 21:44:45.304853 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:45.304838    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0bb417c-a570-4cc1-9d37-e1655e479cdf" path="/var/lib/kubelet/pods/c0bb417c-a570-4cc1-9d37-e1655e479cdf/volumes"
Jul 29 21:44:46.927776 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:46.927737    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:44:55.942738 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:44:55.942651    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:45:04.590656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:45:04.590606    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:45:18.939150 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:45:18.939118    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:45:32.461332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:45:32.461294    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:45:46.637225 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:45:46.637190    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:45:52.741412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:45:52.741365    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:46:00.496680 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:00.496638    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:00.565756 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:00.565716    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:46:02.635248 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:02.635213    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:46:12.245873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:12.245835    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:46:22.646039 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:22.645958    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:46:34.440276 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:34.440243    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:46:44.429319 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:44.429288    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:46:57.335877 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:46:57.335841    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:47:06.233840 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:47:06.233799    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:47:18.536630 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:47:18.536590    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:47:37.727023 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:47:37.726979    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:47:47.051411 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:47:47.051369    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:48:05.430044 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:05.429967    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:48:21.326760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:21.326722    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:48:30.934659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:30.934621    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:48:32.322063 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:32.322021    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:43.726050 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:43.726014    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:48:46.156691 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:46.156659    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:48:58.552858 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:48:58.552822    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:49:07.828456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:07.828424    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:49:08.983228 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:08.983197    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:12.210653 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:12.210615    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:12.317116 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:12.317079    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:12.657012 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:12.656913    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:18.245397 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:18.245357    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:49:18.890370 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:18.890337    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:18.996027 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:18.995989    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:19.094346 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:19.094308    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:19.540200 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:19.540148    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:19.648273 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:19.648234    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:22.139691 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:22.139601    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:22.286007 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:22.285971    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:23.253966 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:23.253930    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:23.392342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:23.392305    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:24.000467 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.000435    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:24.096781 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.096740    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:49:24.147861 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.147828    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"]
Jul 29 21:49:24.148412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.148381    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0bb417c-a570-4cc1-9d37-e1655e479cdf" containerName="main"
Jul 29 21:49:24.148412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.148409    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0bb417c-a570-4cc1-9d37-e1655e479cdf" containerName="main"
Jul 29 21:49:24.148412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.148431    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" containerName="main"
Jul 29 21:49:24.148412 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.148440    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" containerName="main"
Jul 29 21:49:24.148755 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.148559    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bc8435b-7acd-492d-80d8-48cd2a6b7cf8" containerName="main"
Jul 29 21:49:24.148755 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.148575    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0bb417c-a570-4cc1-9d37-e1655e479cdf" containerName="main"
Jul 29 21:49:24.151903 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.151885    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.154532 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.154512    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-972150f2\""
Jul 29 21:49:24.164891 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.164859    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"]
Jul 29 21:49:24.175560 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.175532    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"]
Jul 29 21:49:24.179209 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.179162    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.181777 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.181760    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-972150f2-dockercfg-c29k6\""
Jul 29 21:49:24.194912 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.194887    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"]
Jul 29 21:49:24.240276 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.240242    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j8v5\" (UniqueName: \"kubernetes.io/projected/912166c9-9dfe-4517-8798-fd54b0451bb0-kube-api-access-5j8v5\") pod \"payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.240549 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.240292    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfnjn\" (UniqueName: \"kubernetes.io/projected/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-kube-api-access-kfnjn\") pod \"payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") " pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.240549 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.240336    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/912166c9-9dfe-4517-8798-fd54b0451bb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.240549 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.240407    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-plugins-config-volume\") pod \"payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") " pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.341254 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.341137    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j8v5\" (UniqueName: \"kubernetes.io/projected/912166c9-9dfe-4517-8798-fd54b0451bb0-kube-api-access-5j8v5\") pod \"payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.341254 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.341199    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfnjn\" (UniqueName: \"kubernetes.io/projected/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-kube-api-access-kfnjn\") pod \"payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") " pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.341254 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.341245    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/912166c9-9dfe-4517-8798-fd54b0451bb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.341698 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.341275    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-plugins-config-volume\") pod \"payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") " pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.342212 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.342190    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/912166c9-9dfe-4517-8798-fd54b0451bb0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.342258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.342187    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-plugins-config-volume\") pod \"payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") " pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.350773 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.350745    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfnjn\" (UniqueName: \"kubernetes.io/projected/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-kube-api-access-kfnjn\") pod \"payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") " pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.350873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.350814    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j8v5\" (UniqueName: \"kubernetes.io/projected/912166c9-9dfe-4517-8798-fd54b0451bb0-kube-api-access-5j8v5\") pod \"payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.462303 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.462273    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:24.490478 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.490450    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:24.623023 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.622986    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"]
Jul 29 21:49:24.625275 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:49:24.625242    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029b5c1a_8c3b_4db1_942e_a90ecfaaa3bd.slice/crio-e69556cb3f595986c86bba591a40e31fd3f3c7267e75329de7038bc4d83a868f WatchSource:0}: Error finding container e69556cb3f595986c86bba591a40e31fd3f3c7267e75329de7038bc4d83a868f: Status 404 returned error can't find the container with id e69556cb3f595986c86bba591a40e31fd3f3c7267e75329de7038bc4d83a868f
Jul 29 21:49:24.627134 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.627114    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 21:49:24.652833 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.652809    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"]
Jul 29 21:49:24.654741 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:49:24.654718    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912166c9_9dfe_4517_8798_fd54b0451bb0.slice/crio-5548a812f25ae09ce1738f0d803219c854ff555bde767d603b376845f05c4f8e WatchSource:0}: Error finding container 5548a812f25ae09ce1738f0d803219c854ff555bde767d603b376845f05c4f8e: Status 404 returned error can't find the container with id 5548a812f25ae09ce1738f0d803219c854ff555bde767d603b376845f05c4f8e
Jul 29 21:49:24.886484 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.886389    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4" event={"ID":"912166c9-9dfe-4517-8798-fd54b0451bb0","Type":"ContainerStarted","Data":"5548a812f25ae09ce1738f0d803219c854ff555bde767d603b376845f05c4f8e"}
Jul 29 21:49:24.887621 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:24.887596    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5" event={"ID":"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd","Type":"ContainerStarted","Data":"e69556cb3f595986c86bba591a40e31fd3f3c7267e75329de7038bc4d83a868f"}
Jul 29 21:49:26.902982 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:26.902946    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4" event={"ID":"912166c9-9dfe-4517-8798-fd54b0451bb0","Type":"ContainerStarted","Data":"3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831"}
Jul 29 21:49:26.903406 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:26.903041    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:49:26.904555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:26.904528    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5" event={"ID":"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd","Type":"ContainerStarted","Data":"4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023"}
Jul 29 21:49:26.904682 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:26.904664    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:26.923616 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:26.923576    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4" podStartSLOduration=0.894196595 podStartE2EDuration="2.923564276s" totalImagesPullingTime="2.029367681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:49:26.922655734 +0000 UTC m=+1978.139433208" watchObservedRunningTime="2026-07-29 21:49:26.923564276 +0000 UTC m=+1978.140341772"
Jul 29 21:49:26.941322 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:26.941255    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5" podStartSLOduration=0.875339467 podStartE2EDuration="2.941243841s" totalImagesPullingTime="2.065904374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:49:26.940227132 +0000 UTC m=+1978.157004596" watchObservedRunningTime="2026-07-29 21:49:26.941243841 +0000 UTC m=+1978.158021303"
Jul 29 21:49:30.032184 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:30.032136    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:49:37.912627 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:37.912589    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:49:37.913097 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:49:37.912683    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:50:03.980820 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:03.980781    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:04.019286 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.019253    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"]
Jul 29 21:50:04.019543 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.019515    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5" podUID="029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" containerName="payload-processing" containerID="cri-o://4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023" gracePeriod=30
Jul 29 21:50:04.031842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.031810    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"]
Jul 29 21:50:04.032122 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.032087    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4" podUID="912166c9-9dfe-4517-8798-fd54b0451bb0" containerName="payload-pre-processing" containerID="cri-o://3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831" gracePeriod=30
Jul 29 21:50:04.069969 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.069930    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:04.152297 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.152268    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:04.220498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.220438    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:04.292862 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.292834    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:50:04.320647 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.320620    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:50:04.387408 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.387377    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-plugins-config-volume\") pod \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") "
Jul 29 21:50:04.387753 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.387724    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" (UID: "029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:04.447226 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.447195    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:04.488062 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.488026    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j8v5\" (UniqueName: \"kubernetes.io/projected/912166c9-9dfe-4517-8798-fd54b0451bb0-kube-api-access-5j8v5\") pod \"912166c9-9dfe-4517-8798-fd54b0451bb0\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") "
Jul 29 21:50:04.488281 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.488134    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/912166c9-9dfe-4517-8798-fd54b0451bb0-plugins-config-volume\") pod \"912166c9-9dfe-4517-8798-fd54b0451bb0\" (UID: \"912166c9-9dfe-4517-8798-fd54b0451bb0\") "
Jul 29 21:50:04.488281 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.488195    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kfnjn\" (UniqueName: \"kubernetes.io/projected/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-kube-api-access-kfnjn\") pod \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\" (UID: \"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd\") "
Jul 29 21:50:04.488512 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.488492    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:04.488566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.488522    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/912166c9-9dfe-4517-8798-fd54b0451bb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "912166c9-9dfe-4517-8798-fd54b0451bb0" (UID: "912166c9-9dfe-4517-8798-fd54b0451bb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:04.490357 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.490332    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/912166c9-9dfe-4517-8798-fd54b0451bb0-kube-api-access-5j8v5" (OuterVolumeSpecName: "kube-api-access-5j8v5") pod "912166c9-9dfe-4517-8798-fd54b0451bb0" (UID: "912166c9-9dfe-4517-8798-fd54b0451bb0"). InnerVolumeSpecName "kube-api-access-5j8v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:04.490462 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.490363    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-kube-api-access-kfnjn" (OuterVolumeSpecName: "kube-api-access-kfnjn") pod "029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" (UID: "029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd"). InnerVolumeSpecName "kube-api-access-kfnjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:04.589472 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.589385    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/912166c9-9dfe-4517-8798-fd54b0451bb0-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:04.589472 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.589416    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfnjn\" (UniqueName: \"kubernetes.io/projected/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd-kube-api-access-kfnjn\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:04.589472 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.589426    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j8v5\" (UniqueName: \"kubernetes.io/projected/912166c9-9dfe-4517-8798-fd54b0451bb0-kube-api-access-5j8v5\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:04.664116 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.664078    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:04.995952 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:04.995915    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:05.096082 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.096051    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:05.118679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.118643    2572 generic.go:356] "Generic (PLEG): container finished" podID="912166c9-9dfe-4517-8798-fd54b0451bb0" containerID="3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831" exitCode=0
Jul 29 21:50:05.118864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.118690    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"
Jul 29 21:50:05.118864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.118723    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4" event={"ID":"912166c9-9dfe-4517-8798-fd54b0451bb0","Type":"ContainerDied","Data":"3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831"}
Jul 29 21:50:05.118864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.118787    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4" event={"ID":"912166c9-9dfe-4517-8798-fd54b0451bb0","Type":"ContainerDied","Data":"5548a812f25ae09ce1738f0d803219c854ff555bde767d603b376845f05c4f8e"}
Jul 29 21:50:05.118864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.118824    2572 scope.go:117] "RemoveContainer" containerID="3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831"
Jul 29 21:50:05.120347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.120104    2572 generic.go:356] "Generic (PLEG): container finished" podID="029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" containerID="4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023" exitCode=0
Jul 29 21:50:05.120347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.120153    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"
Jul 29 21:50:05.120347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.120197    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5" event={"ID":"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd","Type":"ContainerDied","Data":"4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023"}
Jul 29 21:50:05.120347 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.120242    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5" event={"ID":"029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd","Type":"ContainerDied","Data":"e69556cb3f595986c86bba591a40e31fd3f3c7267e75329de7038bc4d83a868f"}
Jul 29 21:50:05.128654 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.128630    2572 scope.go:117] "RemoveContainer" containerID="3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831"
Jul 29 21:50:05.128966 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:50:05.128944    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831\": container with ID starting with 3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831 not found: ID does not exist" containerID="3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831"
Jul 29 21:50:05.129019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.128977    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831"} err="failed to get container status \"3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831\": rpc error: code = NotFound desc = could not find container \"3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831\": container with ID starting with 3a1db048be2d342d91b7cca70d7c48b85439ba54afab4a12a193a1fe43442831 not found: ID does not exist"
Jul 29 21:50:05.129019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.129002    2572 scope.go:117] "RemoveContainer" containerID="4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023"
Jul 29 21:50:05.137614 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.137596    2572 scope.go:117] "RemoveContainer" containerID="4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023"
Jul 29 21:50:05.137930 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:50:05.137911    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023\": container with ID starting with 4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023 not found: ID does not exist" containerID="4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023"
Jul 29 21:50:05.137976 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.137938    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023"} err="failed to get container status \"4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023\": rpc error: code = NotFound desc = could not find container \"4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023\": container with ID starting with 4c1d030b3d76eefe019977c447c8fd63038191f0479758fd49364436bd973023 not found: ID does not exist"
Jul 29 21:50:05.150271 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.150236    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"]
Jul 29 21:50:05.159799 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.153779    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-972150f2-d4d4d946d-z6kd5"]
Jul 29 21:50:05.165987 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.165956    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"]
Jul 29 21:50:05.169270 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.169243    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-972150f2-6b5dd6f7b8-7s8r4"]
Jul 29 21:50:05.197405 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.197366    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:05.304716 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.304625    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" path="/var/lib/kubelet/pods/029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd/volumes"
Jul 29 21:50:05.305102 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.305082    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="912166c9-9dfe-4517-8798-fd54b0451bb0" path="/var/lib/kubelet/pods/912166c9-9dfe-4517-8798-fd54b0451bb0/volumes"
Jul 29 21:50:05.568313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.566766    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:05.712373 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:05.712333    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:07.937399 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:07.937368    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:50:08.435939 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:08.435904    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:08.671142 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:08.671084    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:09.052488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.052452    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:09.247877 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.247834    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:09.343672 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.343581    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:09.371124 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371081    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"]
Jul 29 21:50:09.371555 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371541    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" containerName="payload-processing"
Jul 29 21:50:09.371603 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371557    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" containerName="payload-processing"
Jul 29 21:50:09.371603 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371573    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="912166c9-9dfe-4517-8798-fd54b0451bb0" containerName="payload-pre-processing"
Jul 29 21:50:09.371603 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371582    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="912166c9-9dfe-4517-8798-fd54b0451bb0" containerName="payload-pre-processing"
Jul 29 21:50:09.371701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371650    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="912166c9-9dfe-4517-8798-fd54b0451bb0" containerName="payload-pre-processing"
Jul 29 21:50:09.371701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.371665    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="029b5c1a-8c3b-4db1-942e-a90ecfaaa3bd" containerName="payload-processing"
Jul 29 21:50:09.376266 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.376242    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.378936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.378910    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-9394560f-dockercfg-t896r\""
Jul 29 21:50:09.379101 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.378910    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9394560f\""
Jul 29 21:50:09.387571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.387548    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"]
Jul 29 21:50:09.419620 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.419580    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"]
Jul 29 21:50:09.422978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.422959    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.431617 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.431590    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d08364a-42de-4daf-b433-6e9bfe1cc240-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.431757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.431634    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnb2v\" (UniqueName: \"kubernetes.io/projected/9d08364a-42de-4daf-b433-6e9bfe1cc240-kube-api-access-hnb2v\") pod \"payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.431757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.431693    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2094007d-a806-4caa-b87d-3b238eaf6325-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") " pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.431757 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.431715    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vz5xg\" (UniqueName: \"kubernetes.io/projected/2094007d-a806-4caa-b87d-3b238eaf6325-kube-api-access-vz5xg\") pod \"payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") " pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.440958 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.440925    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"]
Jul 29 21:50:09.532498 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.532467    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d08364a-42de-4daf-b433-6e9bfe1cc240-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.532674 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.532515    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnb2v\" (UniqueName: \"kubernetes.io/projected/9d08364a-42de-4daf-b433-6e9bfe1cc240-kube-api-access-hnb2v\") pod \"payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.532674 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.532555    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2094007d-a806-4caa-b87d-3b238eaf6325-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") " pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.532674 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.532572    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vz5xg\" (UniqueName: \"kubernetes.io/projected/2094007d-a806-4caa-b87d-3b238eaf6325-kube-api-access-vz5xg\") pod \"payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") " pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.533300 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.533279    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d08364a-42de-4daf-b433-6e9bfe1cc240-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.533353 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.533321    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2094007d-a806-4caa-b87d-3b238eaf6325-plugins-config-volume\") pod \"payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") " pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.541099 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.541075    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vz5xg\" (UniqueName: \"kubernetes.io/projected/2094007d-a806-4caa-b87d-3b238eaf6325-kube-api-access-vz5xg\") pod \"payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") " pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.541652 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.541624    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnb2v\" (UniqueName: \"kubernetes.io/projected/9d08364a-42de-4daf-b433-6e9bfe1cc240-kube-api-access-hnb2v\") pod \"payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.688414 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.688377    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:09.734199 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.734130    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:09.859360 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.859329    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"]
Jul 29 21:50:09.862435 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:50:09.862401    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2094007d_a806_4caa_b87d_3b238eaf6325.slice/crio-17a6f99b3a746f008ff8d5768d07d0382717734467341e32a5353f4467655f54 WatchSource:0}: Error finding container 17a6f99b3a746f008ff8d5768d07d0382717734467341e32a5353f4467655f54: Status 404 returned error can't find the container with id 17a6f99b3a746f008ff8d5768d07d0382717734467341e32a5353f4467655f54
Jul 29 21:50:09.921906 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:09.921881    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"]
Jul 29 21:50:09.924356 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:50:09.924312    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d08364a_42de_4daf_b433_6e9bfe1cc240.slice/crio-df7e2efc2a679e6bdbd86366daeff4e1d634ec1c8ec8f39e8ec562a22604eaae WatchSource:0}: Error finding container df7e2efc2a679e6bdbd86366daeff4e1d634ec1c8ec8f39e8ec562a22604eaae: Status 404 returned error can't find the container with id df7e2efc2a679e6bdbd86366daeff4e1d634ec1c8ec8f39e8ec562a22604eaae
Jul 29 21:50:10.154653 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.154617    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455" event={"ID":"9d08364a-42de-4daf-b433-6e9bfe1cc240","Type":"ContainerStarted","Data":"c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76"}
Jul 29 21:50:10.155200 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.154670    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455" event={"ID":"9d08364a-42de-4daf-b433-6e9bfe1cc240","Type":"ContainerStarted","Data":"df7e2efc2a679e6bdbd86366daeff4e1d634ec1c8ec8f39e8ec562a22604eaae"}
Jul 29 21:50:10.155200 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.154826    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:10.156371 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.156349    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt" event={"ID":"2094007d-a806-4caa-b87d-3b238eaf6325","Type":"ContainerStarted","Data":"ef90be76950f7077d72b72106f7aa524c2bedbdf79a0da49001eb6edf3aa617e"}
Jul 29 21:50:10.156456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.156394    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:10.156456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.156410    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt" event={"ID":"2094007d-a806-4caa-b87d-3b238eaf6325","Type":"ContainerStarted","Data":"17a6f99b3a746f008ff8d5768d07d0382717734467341e32a5353f4467655f54"}
Jul 29 21:50:10.176389 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.176324    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455" podStartSLOduration=1.17630459 podStartE2EDuration="1.17630459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:10.173309559 +0000 UTC m=+2021.390087024" watchObservedRunningTime="2026-07-29 21:50:10.17630459 +0000 UTC m=+2021.393082053"
Jul 29 21:50:10.193025 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:10.192934    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt" podStartSLOduration=1.192919296 podStartE2EDuration="1.192919296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:10.190574133 +0000 UTC m=+2021.407351596" watchObservedRunningTime="2026-07-29 21:50:10.192919296 +0000 UTC m=+2021.409696757"
Jul 29 21:50:18.629628 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:18.629590    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:50:21.163951 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:21.163918    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:21.164418 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:21.163992    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:27.240665 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:27.240625    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.218189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.218135    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"]
Jul 29 21:50:40.218800 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.218438    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt" podUID="2094007d-a806-4caa-b87d-3b238eaf6325" containerName="payload-processing" containerID="cri-o://ef90be76950f7077d72b72106f7aa524c2bedbdf79a0da49001eb6edf3aa617e" gracePeriod=30
Jul 29 21:50:40.226521 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.226387    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.234543 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.234515    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"]
Jul 29 21:50:40.234762 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.234721    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455" podUID="9d08364a-42de-4daf-b433-6e9bfe1cc240" containerName="payload-pre-processing" containerID="cri-o://c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76" gracePeriod=30
Jul 29 21:50:40.313881 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.313842    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.340991 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.340955    2572 generic.go:356] "Generic (PLEG): container finished" podID="2094007d-a806-4caa-b87d-3b238eaf6325" containerID="ef90be76950f7077d72b72106f7aa524c2bedbdf79a0da49001eb6edf3aa617e" exitCode=0
Jul 29 21:50:40.341152 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.341006    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt" event={"ID":"2094007d-a806-4caa-b87d-3b238eaf6325","Type":"ContainerDied","Data":"ef90be76950f7077d72b72106f7aa524c2bedbdf79a0da49001eb6edf3aa617e"}
Jul 29 21:50:40.414214 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.414158    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.520078 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.520004    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:40.523782 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.523758    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:40.621452 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621417    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2094007d-a806-4caa-b87d-3b238eaf6325-plugins-config-volume\") pod \"2094007d-a806-4caa-b87d-3b238eaf6325\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") "
Jul 29 21:50:40.621664 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621472    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnb2v\" (UniqueName: \"kubernetes.io/projected/9d08364a-42de-4daf-b433-6e9bfe1cc240-kube-api-access-hnb2v\") pod \"9d08364a-42de-4daf-b433-6e9bfe1cc240\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") "
Jul 29 21:50:40.621664 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621511    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d08364a-42de-4daf-b433-6e9bfe1cc240-plugins-config-volume\") pod \"9d08364a-42de-4daf-b433-6e9bfe1cc240\" (UID: \"9d08364a-42de-4daf-b433-6e9bfe1cc240\") "
Jul 29 21:50:40.621664 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621601    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vz5xg\" (UniqueName: \"kubernetes.io/projected/2094007d-a806-4caa-b87d-3b238eaf6325-kube-api-access-vz5xg\") pod \"2094007d-a806-4caa-b87d-3b238eaf6325\" (UID: \"2094007d-a806-4caa-b87d-3b238eaf6325\") "
Jul 29 21:50:40.621884 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621854    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2094007d-a806-4caa-b87d-3b238eaf6325-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2094007d-a806-4caa-b87d-3b238eaf6325" (UID: "2094007d-a806-4caa-b87d-3b238eaf6325"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:40.621941 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621900    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d08364a-42de-4daf-b433-6e9bfe1cc240-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d08364a-42de-4daf-b433-6e9bfe1cc240" (UID: "9d08364a-42de-4daf-b433-6e9bfe1cc240"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:50:40.622010 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621980    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d08364a-42de-4daf-b433-6e9bfe1cc240-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:40.622010 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.621996    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2094007d-a806-4caa-b87d-3b238eaf6325-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:40.623771 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.623747    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d08364a-42de-4daf-b433-6e9bfe1cc240-kube-api-access-hnb2v" (OuterVolumeSpecName: "kube-api-access-hnb2v") pod "9d08364a-42de-4daf-b433-6e9bfe1cc240" (UID: "9d08364a-42de-4daf-b433-6e9bfe1cc240"). InnerVolumeSpecName "kube-api-access-hnb2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:40.624099 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.624078    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2094007d-a806-4caa-b87d-3b238eaf6325-kube-api-access-vz5xg" (OuterVolumeSpecName: "kube-api-access-vz5xg") pod "2094007d-a806-4caa-b87d-3b238eaf6325" (UID: "2094007d-a806-4caa-b87d-3b238eaf6325"). InnerVolumeSpecName "kube-api-access-vz5xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:50:40.667767 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.667721    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:40.723229 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.723161    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnb2v\" (UniqueName: \"kubernetes.io/projected/9d08364a-42de-4daf-b433-6e9bfe1cc240-kube-api-access-hnb2v\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:40.723229 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.723227    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vz5xg\" (UniqueName: \"kubernetes.io/projected/2094007d-a806-4caa-b87d-3b238eaf6325-kube-api-access-vz5xg\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:50:40.839732 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:40.839641    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:41.087934 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.087900    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:41.348331 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.348301    2572 generic.go:356] "Generic (PLEG): container finished" podID="9d08364a-42de-4daf-b433-6e9bfe1cc240" containerID="c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76" exitCode=0
Jul 29 21:50:41.348770 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.348345    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"
Jul 29 21:50:41.348770 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.348382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455" event={"ID":"9d08364a-42de-4daf-b433-6e9bfe1cc240","Type":"ContainerDied","Data":"c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76"}
Jul 29 21:50:41.348770 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.348433    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455" event={"ID":"9d08364a-42de-4daf-b433-6e9bfe1cc240","Type":"ContainerDied","Data":"df7e2efc2a679e6bdbd86366daeff4e1d634ec1c8ec8f39e8ec562a22604eaae"}
Jul 29 21:50:41.348770 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.348462    2572 scope.go:117] "RemoveContainer" containerID="c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76"
Jul 29 21:50:41.349788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.349773    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"
Jul 29 21:50:41.349873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.349770    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt" event={"ID":"2094007d-a806-4caa-b87d-3b238eaf6325","Type":"ContainerDied","Data":"17a6f99b3a746f008ff8d5768d07d0382717734467341e32a5353f4467655f54"}
Jul 29 21:50:41.358017 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.357984    2572 scope.go:117] "RemoveContainer" containerID="c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76"
Jul 29 21:50:41.358426 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:50:41.358403    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76\": container with ID starting with c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76 not found: ID does not exist" containerID="c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76"
Jul 29 21:50:41.358540 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.358442    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76"} err="failed to get container status \"c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76\": rpc error: code = NotFound desc = could not find container \"c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76\": container with ID starting with c21cb56e158619b345bcccb343f9f569c0cb2b7ca1f4ce96f26aa72bd16dce76 not found: ID does not exist"
Jul 29 21:50:41.358540 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.358460    2572 scope.go:117] "RemoveContainer" containerID="ef90be76950f7077d72b72106f7aa524c2bedbdf79a0da49001eb6edf3aa617e"
Jul 29 21:50:41.370199 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.370159    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"]
Jul 29 21:50:41.376380 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.376334    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-9394560f-66c78594fc-79ttt"]
Jul 29 21:50:41.396071 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.396032    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"]
Jul 29 21:50:41.402690 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.402660    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9394560f-8b996954d-qd455"]
Jul 29 21:50:41.454493 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.454464    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:41.586960 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:41.586864    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:43.304907 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:43.304869    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2094007d-a806-4caa-b87d-3b238eaf6325" path="/var/lib/kubelet/pods/2094007d-a806-4caa-b87d-3b238eaf6325/volumes"
Jul 29 21:50:43.305313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:43.305251    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d08364a-42de-4daf-b433-6e9bfe1cc240" path="/var/lib/kubelet/pods/9d08364a-42de-4daf-b433-6e9bfe1cc240/volumes"
Jul 29 21:50:43.519137 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:43.519101    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:43.618112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:43.618017    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:44.125701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:44.125660    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:50:44.670792 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:44.670754    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:44.966636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:44.966594    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:45.411391 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.411309    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:45.512361 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.512326    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:50:45.570766 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.570727    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"]
Jul 29 21:50:45.571189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.571152    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d08364a-42de-4daf-b433-6e9bfe1cc240" containerName="payload-pre-processing"
Jul 29 21:50:45.571189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.571179    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d08364a-42de-4daf-b433-6e9bfe1cc240" containerName="payload-pre-processing"
Jul 29 21:50:45.571342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.571197    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2094007d-a806-4caa-b87d-3b238eaf6325" containerName="payload-processing"
Jul 29 21:50:45.571342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.571209    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2094007d-a806-4caa-b87d-3b238eaf6325" containerName="payload-processing"
Jul 29 21:50:45.571342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.571304    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2094007d-a806-4caa-b87d-3b238eaf6325" containerName="payload-processing"
Jul 29 21:50:45.571342 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.571315    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d08364a-42de-4daf-b433-6e9bfe1cc240" containerName="payload-pre-processing"
Jul 29 21:50:45.575970 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.575941    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.579133 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.579106    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0856dcaa\""
Jul 29 21:50:45.579287 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.579113    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0856dcaa-dockercfg-7f52r\""
Jul 29 21:50:45.589055 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.589023    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"]
Jul 29 21:50:45.616455 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.616415    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"]
Jul 29 21:50:45.620056 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.620038    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.634632 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.634604    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"]
Jul 29 21:50:45.676001 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.675907    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vcmt\" (UniqueName: \"kubernetes.io/projected/5ee69743-e3d3-4246-8f32-28f56ed3227a-kube-api-access-8vcmt\") pod \"payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.676001 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.675982    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ee69743-e3d3-4246-8f32-28f56ed3227a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.776908 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.776867    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vcmt\" (UniqueName: \"kubernetes.io/projected/5ee69743-e3d3-4246-8f32-28f56ed3227a-kube-api-access-8vcmt\") pod \"payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.777120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.776928    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428781cc-8a13-4aa8-ba36-4234d324eaf7-plugins-config-volume\") pod \"payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") " pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.777120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.777035    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ee69743-e3d3-4246-8f32-28f56ed3227a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.777120 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.777076    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7lvn\" (UniqueName: \"kubernetes.io/projected/428781cc-8a13-4aa8-ba36-4234d324eaf7-kube-api-access-n7lvn\") pod \"payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") " pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.777767 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.777744    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ee69743-e3d3-4246-8f32-28f56ed3227a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.785666 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.785636    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vcmt\" (UniqueName: \"kubernetes.io/projected/5ee69743-e3d3-4246-8f32-28f56ed3227a-kube-api-access-8vcmt\") pod \"payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:45.878379 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.878342    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7lvn\" (UniqueName: \"kubernetes.io/projected/428781cc-8a13-4aa8-ba36-4234d324eaf7-kube-api-access-n7lvn\") pod \"payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") " pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.878596 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.878416    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428781cc-8a13-4aa8-ba36-4234d324eaf7-plugins-config-volume\") pod \"payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") " pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.879110 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.879088    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428781cc-8a13-4aa8-ba36-4234d324eaf7-plugins-config-volume\") pod \"payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") " pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.886936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.886898    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7lvn\" (UniqueName: \"kubernetes.io/projected/428781cc-8a13-4aa8-ba36-4234d324eaf7-kube-api-access-n7lvn\") pod \"payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") " pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.887969 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.887948    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:45.930637 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:45.930586    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:46.042683 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.042608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"]
Jul 29 21:50:46.045988 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:50:46.045932    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod428781cc_8a13_4aa8_ba36_4234d324eaf7.slice/crio-e5c03b2ad52442819a7b908b2ab1354501b17db84f07bf216e0f231180c01c15 WatchSource:0}: Error finding container e5c03b2ad52442819a7b908b2ab1354501b17db84f07bf216e0f231180c01c15: Status 404 returned error can't find the container with id e5c03b2ad52442819a7b908b2ab1354501b17db84f07bf216e0f231180c01c15
Jul 29 21:50:46.100019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.099992    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"]
Jul 29 21:50:46.101502 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:50:46.101472    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ee69743_e3d3_4246_8f32_28f56ed3227a.slice/crio-8c4801cc7a8ef27a64bfb4429e041292a2f83ed7d19b530152d95fba89fc286d WatchSource:0}: Error finding container 8c4801cc7a8ef27a64bfb4429e041292a2f83ed7d19b530152d95fba89fc286d: Status 404 returned error can't find the container with id 8c4801cc7a8ef27a64bfb4429e041292a2f83ed7d19b530152d95fba89fc286d
Jul 29 21:50:46.384649 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.384567    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v" event={"ID":"428781cc-8a13-4aa8-ba36-4234d324eaf7","Type":"ContainerStarted","Data":"585100233a4303458467ea87622f9fd431b6f1bbcf33a9b9ecd51e8972820e05"}
Jul 29 21:50:46.384649 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.384627    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v" event={"ID":"428781cc-8a13-4aa8-ba36-4234d324eaf7","Type":"ContainerStarted","Data":"e5c03b2ad52442819a7b908b2ab1354501b17db84f07bf216e0f231180c01c15"}
Jul 29 21:50:46.384873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.384658    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:46.386244 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.386208    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb" event={"ID":"5ee69743-e3d3-4246-8f32-28f56ed3227a","Type":"ContainerStarted","Data":"90cd10a360ece8292bf5cc9e6553ddffa731f8166fbe14af4dfcb68f01610d64"}
Jul 29 21:50:46.386350 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.386269    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:50:46.386350 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.386294    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb" event={"ID":"5ee69743-e3d3-4246-8f32-28f56ed3227a","Type":"ContainerStarted","Data":"8c4801cc7a8ef27a64bfb4429e041292a2f83ed7d19b530152d95fba89fc286d"}
Jul 29 21:50:46.403636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.403591    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v" podStartSLOduration=1.403575503 podStartE2EDuration="1.403575503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:46.401265867 +0000 UTC m=+2057.618043329" watchObservedRunningTime="2026-07-29 21:50:46.403575503 +0000 UTC m=+2057.620352965"
Jul 29 21:50:46.419025 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:46.418978    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb" podStartSLOduration=1.418964873 podStartE2EDuration="1.418964873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:50:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:50:46.416670966 +0000 UTC m=+2057.633448429" watchObservedRunningTime="2026-07-29 21:50:46.418964873 +0000 UTC m=+2057.635742334"
Jul 29 21:50:54.929258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:54.929156    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:50:57.393128 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:57.393094    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:50:57.393552 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:50:57.393186    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:51:03.779601 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:03.779554    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:15.379525 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.379482    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:15.406734 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.406696    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"]
Jul 29 21:51:15.407065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.407009    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v" podUID="428781cc-8a13-4aa8-ba36-4234d324eaf7" containerName="payload-processing" containerID="cri-o://585100233a4303458467ea87622f9fd431b6f1bbcf33a9b9ecd51e8972820e05" gracePeriod=30
Jul 29 21:51:15.413712 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.413684    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"]
Jul 29 21:51:15.414121 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.414092    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb" podUID="5ee69743-e3d3-4246-8f32-28f56ed3227a" containerName="payload-pre-processing" containerID="cri-o://90cd10a360ece8292bf5cc9e6553ddffa731f8166fbe14af4dfcb68f01610d64" gracePeriod=30
Jul 29 21:51:15.455482 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.455446    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:15.532636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.532607    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:15.555723 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.555684    2572 generic.go:356] "Generic (PLEG): container finished" podID="5ee69743-e3d3-4246-8f32-28f56ed3227a" containerID="90cd10a360ece8292bf5cc9e6553ddffa731f8166fbe14af4dfcb68f01610d64" exitCode=0
Jul 29 21:51:15.555891 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.555751    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb" event={"ID":"5ee69743-e3d3-4246-8f32-28f56ed3227a","Type":"ContainerDied","Data":"90cd10a360ece8292bf5cc9e6553ddffa731f8166fbe14af4dfcb68f01610d64"}
Jul 29 21:51:15.557521 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.557498    2572 generic.go:356] "Generic (PLEG): container finished" podID="428781cc-8a13-4aa8-ba36-4234d324eaf7" containerID="585100233a4303458467ea87622f9fd431b6f1bbcf33a9b9ecd51e8972820e05" exitCode=0
Jul 29 21:51:15.557657 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.557560    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v" event={"ID":"428781cc-8a13-4aa8-ba36-4234d324eaf7","Type":"ContainerDied","Data":"585100233a4303458467ea87622f9fd431b6f1bbcf33a9b9ecd51e8972820e05"}
Jul 29 21:51:15.622308 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.622276    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:15.699909 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.699884    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:51:15.702663 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.702643    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:51:15.851695 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.851655    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428781cc-8a13-4aa8-ba36-4234d324eaf7-plugins-config-volume\") pod \"428781cc-8a13-4aa8-ba36-4234d324eaf7\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") "
Jul 29 21:51:15.851904 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.851747    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ee69743-e3d3-4246-8f32-28f56ed3227a-plugins-config-volume\") pod \"5ee69743-e3d3-4246-8f32-28f56ed3227a\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") "
Jul 29 21:51:15.851904 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.851769    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7lvn\" (UniqueName: \"kubernetes.io/projected/428781cc-8a13-4aa8-ba36-4234d324eaf7-kube-api-access-n7lvn\") pod \"428781cc-8a13-4aa8-ba36-4234d324eaf7\" (UID: \"428781cc-8a13-4aa8-ba36-4234d324eaf7\") "
Jul 29 21:51:15.851904 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.851794    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vcmt\" (UniqueName: \"kubernetes.io/projected/5ee69743-e3d3-4246-8f32-28f56ed3227a-kube-api-access-8vcmt\") pod \"5ee69743-e3d3-4246-8f32-28f56ed3227a\" (UID: \"5ee69743-e3d3-4246-8f32-28f56ed3227a\") "
Jul 29 21:51:15.852092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.852063    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/428781cc-8a13-4aa8-ba36-4234d324eaf7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "428781cc-8a13-4aa8-ba36-4234d324eaf7" (UID: "428781cc-8a13-4aa8-ba36-4234d324eaf7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:51:15.852240 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.852219    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ee69743-e3d3-4246-8f32-28f56ed3227a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ee69743-e3d3-4246-8f32-28f56ed3227a" (UID: "5ee69743-e3d3-4246-8f32-28f56ed3227a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:51:15.854039 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.854007    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428781cc-8a13-4aa8-ba36-4234d324eaf7-kube-api-access-n7lvn" (OuterVolumeSpecName: "kube-api-access-n7lvn") pod "428781cc-8a13-4aa8-ba36-4234d324eaf7" (UID: "428781cc-8a13-4aa8-ba36-4234d324eaf7"). InnerVolumeSpecName "kube-api-access-n7lvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:51:15.854039 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.854036    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ee69743-e3d3-4246-8f32-28f56ed3227a-kube-api-access-8vcmt" (OuterVolumeSpecName: "kube-api-access-8vcmt") pod "5ee69743-e3d3-4246-8f32-28f56ed3227a" (UID: "5ee69743-e3d3-4246-8f32-28f56ed3227a"). InnerVolumeSpecName "kube-api-access-8vcmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:51:15.953417 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.953384    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ee69743-e3d3-4246-8f32-28f56ed3227a-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:15.953417 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.953413    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7lvn\" (UniqueName: \"kubernetes.io/projected/428781cc-8a13-4aa8-ba36-4234d324eaf7-kube-api-access-n7lvn\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:15.953417 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.953423    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vcmt\" (UniqueName: \"kubernetes.io/projected/5ee69743-e3d3-4246-8f32-28f56ed3227a-kube-api-access-8vcmt\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:15.953677 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:15.953431    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428781cc-8a13-4aa8-ba36-4234d324eaf7-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:16.565074 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.565045    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"
Jul 29 21:51:16.565558 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.565074    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v" event={"ID":"428781cc-8a13-4aa8-ba36-4234d324eaf7","Type":"ContainerDied","Data":"e5c03b2ad52442819a7b908b2ab1354501b17db84f07bf216e0f231180c01c15"}
Jul 29 21:51:16.565558 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.565158    2572 scope.go:117] "RemoveContainer" containerID="585100233a4303458467ea87622f9fd431b6f1bbcf33a9b9ecd51e8972820e05"
Jul 29 21:51:16.566659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.566621    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb" event={"ID":"5ee69743-e3d3-4246-8f32-28f56ed3227a","Type":"ContainerDied","Data":"8c4801cc7a8ef27a64bfb4429e041292a2f83ed7d19b530152d95fba89fc286d"}
Jul 29 21:51:16.566659 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.566648    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"
Jul 29 21:51:16.581864 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.581837    2572 scope.go:117] "RemoveContainer" containerID="90cd10a360ece8292bf5cc9e6553ddffa731f8166fbe14af4dfcb68f01610d64"
Jul 29 21:51:16.595583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.595530    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"]
Jul 29 21:51:16.602608 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.602578    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0856dcaa-67d4c9747b-hcr9v"]
Jul 29 21:51:16.613701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.613672    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"]
Jul 29 21:51:16.618580 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.618554    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0856dcaa-5f5876574d-zjwtb"]
Jul 29 21:51:16.865572 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:16.865481    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:17.143775 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:17.143680    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:17.304317 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:17.304282    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="428781cc-8a13-4aa8-ba36-4234d324eaf7" path="/var/lib/kubelet/pods/428781cc-8a13-4aa8-ba36-4234d324eaf7/volumes"
Jul 29 21:51:17.304644 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:17.304631    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ee69743-e3d3-4246-8f32-28f56ed3227a" path="/var/lib/kubelet/pods/5ee69743-e3d3-4246-8f32-28f56ed3227a/volumes"
Jul 29 21:51:17.368940 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:17.368906    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:17.714028 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:17.713995    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:17.875630 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:17.875547    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:19.532074 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:19.532039    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:51:19.691803 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:19.691762    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:19.782798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:19.782711    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:20.810340 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:20.810306    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:20.983780 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:20.983744    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:21.325041 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.325007    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:21.420314 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.420274    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:21.427573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.427537    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"]
Jul 29 21:51:21.427983 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.427956    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ee69743-e3d3-4246-8f32-28f56ed3227a" containerName="payload-pre-processing"
Jul 29 21:51:21.427983 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.427976    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ee69743-e3d3-4246-8f32-28f56ed3227a" containerName="payload-pre-processing"
Jul 29 21:51:21.428265 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.428013    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="428781cc-8a13-4aa8-ba36-4234d324eaf7" containerName="payload-processing"
Jul 29 21:51:21.428265 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.428023    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="428781cc-8a13-4aa8-ba36-4234d324eaf7" containerName="payload-processing"
Jul 29 21:51:21.428265 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.428190    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="428781cc-8a13-4aa8-ba36-4234d324eaf7" containerName="payload-processing"
Jul 29 21:51:21.428265 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.428209    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ee69743-e3d3-4246-8f32-28f56ed3227a" containerName="payload-pre-processing"
Jul 29 21:51:21.433226 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.433162    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.436534 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.436508    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-48a37a48-dockercfg-9kz4q\""
Jul 29 21:51:21.436681 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.436508    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-48a37a48\""
Jul 29 21:51:21.446715 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.446587    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"]
Jul 29 21:51:21.455238 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.455153    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"]
Jul 29 21:51:21.458841 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.458814    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.469549 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.469446    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"]
Jul 29 21:51:21.607937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.607851    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdc92\" (UniqueName: \"kubernetes.io/projected/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-kube-api-access-wdc92\") pod \"payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.607937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.607908    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtx9r\" (UniqueName: \"kubernetes.io/projected/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-kube-api-access-vtx9r\") pod \"payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") " pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.607937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.607932    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.608196 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.607952    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-plugins-config-volume\") pod \"payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") " pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.709090 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.709047    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdc92\" (UniqueName: \"kubernetes.io/projected/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-kube-api-access-wdc92\") pod \"payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.709313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.709113    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtx9r\" (UniqueName: \"kubernetes.io/projected/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-kube-api-access-vtx9r\") pod \"payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") " pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.709313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.709133    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.709313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.709154    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-plugins-config-volume\") pod \"payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") " pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.709935 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.709913    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-plugins-config-volume\") pod \"payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") " pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.709999 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.709922    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.724319 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.724278    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtx9r\" (UniqueName: \"kubernetes.io/projected/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-kube-api-access-vtx9r\") pod \"payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") " pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.726157 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.726128    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdc92\" (UniqueName: \"kubernetes.io/projected/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-kube-api-access-wdc92\") pod \"payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.746998 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.746951    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:21.772579 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.772533    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:21.914074 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.914049    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"]
Jul 29 21:51:21.916634 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:51:21.916607    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fcdbe0f_2992_4e57_a0e5_9c933853bc10.slice/crio-b4f2fe2c3b9e836e5010536f4b8f920f32390bbfa0ab4897af71e0f6090cc009 WatchSource:0}: Error finding container b4f2fe2c3b9e836e5010536f4b8f920f32390bbfa0ab4897af71e0f6090cc009: Status 404 returned error can't find the container with id b4f2fe2c3b9e836e5010536f4b8f920f32390bbfa0ab4897af71e0f6090cc009
Jul 29 21:51:21.944009 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:21.943986    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"]
Jul 29 21:51:21.953318 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:51:21.953289    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b1488ea_4942_49d5_8bc6_53fc709e7b7e.slice/crio-f9fdb0d826c9b7921c1b3accd04586d49d3732fa49b46519118cd25994b6aad2 WatchSource:0}: Error finding container f9fdb0d826c9b7921c1b3accd04586d49d3732fa49b46519118cd25994b6aad2: Status 404 returned error can't find the container with id f9fdb0d826c9b7921c1b3accd04586d49d3732fa49b46519118cd25994b6aad2
Jul 29 21:51:22.603471 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.603437    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99" event={"ID":"3fcdbe0f-2992-4e57-a0e5-9c933853bc10","Type":"ContainerStarted","Data":"a5af53055a15c03b370df1d16a7332177499f92088cd6323555e10bcae7e5161"}
Jul 29 21:51:22.603679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.603503    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:22.603679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.603528    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99" event={"ID":"3fcdbe0f-2992-4e57-a0e5-9c933853bc10","Type":"ContainerStarted","Data":"b4f2fe2c3b9e836e5010536f4b8f920f32390bbfa0ab4897af71e0f6090cc009"}
Jul 29 21:51:22.605025 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.605000    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq" event={"ID":"7b1488ea-4942-49d5-8bc6-53fc709e7b7e","Type":"ContainerStarted","Data":"f657278b75c792c73f34c9518088ba3ec3f49c34ebadb6d74676c09fd7eb9802"}
Jul 29 21:51:22.605145 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.605043    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:22.605145 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.605059    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq" event={"ID":"7b1488ea-4942-49d5-8bc6-53fc709e7b7e","Type":"ContainerStarted","Data":"f9fdb0d826c9b7921c1b3accd04586d49d3732fa49b46519118cd25994b6aad2"}
Jul 29 21:51:22.621137 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.621097    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99" podStartSLOduration=1.621081594 podStartE2EDuration="1.621081594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:51:22.620272124 +0000 UTC m=+2093.837049587" watchObservedRunningTime="2026-07-29 21:51:22.621081594 +0000 UTC m=+2093.837859059"
Jul 29 21:51:22.638214 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:22.638141    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq" podStartSLOduration=1.638122927 podStartE2EDuration="1.638122927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:51:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:51:22.636588173 +0000 UTC m=+2093.853365635" watchObservedRunningTime="2026-07-29 21:51:22.638122927 +0000 UTC m=+2093.854900389"
Jul 29 21:51:30.728641 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:30.728602    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:51:33.613313 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:33.613280    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:33.613720 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:33.613361    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:40.880737 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:40.880692    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:51.481374 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.481334    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:51.513936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.513903    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"]
Jul 29 21:51:51.514191 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.514150    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq" podUID="7b1488ea-4942-49d5-8bc6-53fc709e7b7e" containerName="payload-processing" containerID="cri-o://f657278b75c792c73f34c9518088ba3ec3f49c34ebadb6d74676c09fd7eb9802" gracePeriod=30
Jul 29 21:51:51.526954 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.526916    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"]
Jul 29 21:51:51.527377 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.527341    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99" podUID="3fcdbe0f-2992-4e57-a0e5-9c933853bc10" containerName="payload-pre-processing" containerID="cri-o://a5af53055a15c03b370df1d16a7332177499f92088cd6323555e10bcae7e5161" gracePeriod=30
Jul 29 21:51:51.560614 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.560579    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:51.638003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.637971    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:51.716527 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.716485    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:51.771559 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.771470    2572 generic.go:356] "Generic (PLEG): container finished" podID="7b1488ea-4942-49d5-8bc6-53fc709e7b7e" containerID="f657278b75c792c73f34c9518088ba3ec3f49c34ebadb6d74676c09fd7eb9802" exitCode=0
Jul 29 21:51:51.771719 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.771642    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq" event={"ID":"7b1488ea-4942-49d5-8bc6-53fc709e7b7e","Type":"ContainerDied","Data":"f657278b75c792c73f34c9518088ba3ec3f49c34ebadb6d74676c09fd7eb9802"}
Jul 29 21:51:51.773919 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.773871    2572 generic.go:356] "Generic (PLEG): container finished" podID="3fcdbe0f-2992-4e57-a0e5-9c933853bc10" containerID="a5af53055a15c03b370df1d16a7332177499f92088cd6323555e10bcae7e5161" exitCode=0
Jul 29 21:51:51.774056 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.773925    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99" event={"ID":"3fcdbe0f-2992-4e57-a0e5-9c933853bc10","Type":"ContainerDied","Data":"a5af53055a15c03b370df1d16a7332177499f92088cd6323555e10bcae7e5161"}
Jul 29 21:51:51.839100 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.839069    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:51.842283 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.842259    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:51.845163 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.845131    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:51.990685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.990653    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtx9r\" (UniqueName: \"kubernetes.io/projected/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-kube-api-access-vtx9r\") pod \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") "
Jul 29 21:51:51.990685 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.990688    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-plugins-config-volume\") pod \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\" (UID: \"7b1488ea-4942-49d5-8bc6-53fc709e7b7e\") "
Jul 29 21:51:51.990948 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.990716    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdc92\" (UniqueName: \"kubernetes.io/projected/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-kube-api-access-wdc92\") pod \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") "
Jul 29 21:51:51.990948 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.990737    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-plugins-config-volume\") pod \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\" (UID: \"3fcdbe0f-2992-4e57-a0e5-9c933853bc10\") "
Jul 29 21:51:51.991203 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.991144    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3fcdbe0f-2992-4e57-a0e5-9c933853bc10" (UID: "3fcdbe0f-2992-4e57-a0e5-9c933853bc10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:51:51.991338 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.991164    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b1488ea-4942-49d5-8bc6-53fc709e7b7e" (UID: "7b1488ea-4942-49d5-8bc6-53fc709e7b7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:51:51.992860 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.992836    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-kube-api-access-wdc92" (OuterVolumeSpecName: "kube-api-access-wdc92") pod "3fcdbe0f-2992-4e57-a0e5-9c933853bc10" (UID: "3fcdbe0f-2992-4e57-a0e5-9c933853bc10"). InnerVolumeSpecName "kube-api-access-wdc92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:51:51.992860 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:51.992867    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-kube-api-access-vtx9r" (OuterVolumeSpecName: "kube-api-access-vtx9r") pod "7b1488ea-4942-49d5-8bc6-53fc709e7b7e" (UID: "7b1488ea-4942-49d5-8bc6-53fc709e7b7e"). InnerVolumeSpecName "kube-api-access-vtx9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:51:52.081507 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.081411    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:52.091956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.091924    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtx9r\" (UniqueName: \"kubernetes.io/projected/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-kube-api-access-vtx9r\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:52.091956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.091951    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b1488ea-4942-49d5-8bc6-53fc709e7b7e-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:52.091956 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.091961    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdc92\" (UniqueName: \"kubernetes.io/projected/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-kube-api-access-wdc92\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:52.092222 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.091971    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fcdbe0f-2992-4e57-a0e5-9c933853bc10-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:51:52.563124 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.563089    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:52.643061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.643027    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:52.743057 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.743015    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:51:52.781273 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.781238    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"
Jul 29 21:51:52.781481 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.781243    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq" event={"ID":"7b1488ea-4942-49d5-8bc6-53fc709e7b7e","Type":"ContainerDied","Data":"f9fdb0d826c9b7921c1b3accd04586d49d3732fa49b46519118cd25994b6aad2"}
Jul 29 21:51:52.781481 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.781382    2572 scope.go:117] "RemoveContainer" containerID="f657278b75c792c73f34c9518088ba3ec3f49c34ebadb6d74676c09fd7eb9802"
Jul 29 21:51:52.783019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.782957    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99" event={"ID":"3fcdbe0f-2992-4e57-a0e5-9c933853bc10","Type":"ContainerDied","Data":"b4f2fe2c3b9e836e5010536f4b8f920f32390bbfa0ab4897af71e0f6090cc009"}
Jul 29 21:51:52.783019 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.782983    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"
Jul 29 21:51:52.792325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.792300    2572 scope.go:117] "RemoveContainer" containerID="a5af53055a15c03b370df1d16a7332177499f92088cd6323555e10bcae7e5161"
Jul 29 21:51:52.806718 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.806679    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"]
Jul 29 21:51:52.812534 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.812504    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-48a37a48-6b7b66786-pdflq"]
Jul 29 21:51:52.823699 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.823665    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"]
Jul 29 21:51:52.832450 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:52.832416    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-48a37a48-7747fc7b4-pkf99"]
Jul 29 21:51:53.304215 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:53.304153    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fcdbe0f-2992-4e57-a0e5-9c933853bc10" path="/var/lib/kubelet/pods/3fcdbe0f-2992-4e57-a0e5-9c933853bc10/volumes"
Jul 29 21:51:53.304553 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:53.304539    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b1488ea-4942-49d5-8bc6-53fc709e7b7e" path="/var/lib/kubelet/pods/7b1488ea-4942-49d5-8bc6-53fc709e7b7e/volumes"
Jul 29 21:51:55.529294 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:51:55.529254    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:52:14.691010 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:14.690969    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:16.740038 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:16.736983    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:52:21.245724 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:21.245680    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:21.448873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:21.448829    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:21.546164 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:21.546041    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:21.876516 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:21.876427    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:47.899115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:47.899073    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:54.760679 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:54.760642    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:54.862157 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:54.862119    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:52:54.966214 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:52:54.966161    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:10.889938 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:10.889902    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:16.663348 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:16.663286    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:16.767536 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:16.767487    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:16.862223 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:16.862164    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:27.404813 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:27.404782    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:38.486034 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:38.485996    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:45.234009 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:45.233963    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:45.340980 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:45.340925    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:45.461583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:45.461541    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:45.563092 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:45.563009    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:46.258565 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:46.258531    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:46.356483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:46.356453    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:48.038936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:48.038895    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:50.335979 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:50.335943    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:53:54.504244 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:54.504139    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:54.606987 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:54.606948    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:55.280060 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:55.280015    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:53:58.639063 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:53:58.639030    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:54:01.899693 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:01.899654    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:02.002211 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:02.002149    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:02.155960 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:02.155872    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:02.518387 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:02.518353    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:02.887865 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:02.887776    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:05.427809 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:05.427772    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:05.569950 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:05.569909    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:06.643432 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:06.643401    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:06.882359 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:06.882320    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:07.210433 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.210396    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:07.312065 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.312029    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:07.399027 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.398993    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"]
Jul 29 21:54:07.399433 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.399418    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b1488ea-4942-49d5-8bc6-53fc709e7b7e" containerName="payload-processing"
Jul 29 21:54:07.399487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.399435    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b1488ea-4942-49d5-8bc6-53fc709e7b7e" containerName="payload-processing"
Jul 29 21:54:07.399487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.399447    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fcdbe0f-2992-4e57-a0e5-9c933853bc10" containerName="payload-pre-processing"
Jul 29 21:54:07.399487 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.399456    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fcdbe0f-2992-4e57-a0e5-9c933853bc10" containerName="payload-pre-processing"
Jul 29 21:54:07.399583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.399531    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b1488ea-4942-49d5-8bc6-53fc709e7b7e" containerName="payload-processing"
Jul 29 21:54:07.399583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.399546    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fcdbe0f-2992-4e57-a0e5-9c933853bc10" containerName="payload-pre-processing"
Jul 29 21:54:07.402636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.402621    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.405224 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.405197    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-d5dee3c8\""
Jul 29 21:54:07.405368 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.405238    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-d5dee3c8-dockercfg-c979k\""
Jul 29 21:54:07.417358 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.417330    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"]
Jul 29 21:54:07.473100 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.473020    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"]
Jul 29 21:54:07.476945 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.476921    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.488779 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.488751    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"]
Jul 29 21:54:07.550509 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.550476    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73c6721-8084-4892-a0ff-9e30a7306dd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") " pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.550716 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.550528    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9wzq\" (UniqueName: \"kubernetes.io/projected/a73c6721-8084-4892-a0ff-9e30a7306dd0-kube-api-access-l9wzq\") pod \"payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") " pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.550716 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.550634    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/027af5a6-41ae-4dcd-813f-47dfc3df064b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.550716 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.550679    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fnms\" (UniqueName: \"kubernetes.io/projected/027af5a6-41ae-4dcd-813f-47dfc3df064b-kube-api-access-4fnms\") pod \"payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.651208 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.651146    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/027af5a6-41ae-4dcd-813f-47dfc3df064b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.651693 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.651295    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fnms\" (UniqueName: \"kubernetes.io/projected/027af5a6-41ae-4dcd-813f-47dfc3df064b-kube-api-access-4fnms\") pod \"payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.651693 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.651350    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73c6721-8084-4892-a0ff-9e30a7306dd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") " pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.651693 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.651389    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9wzq\" (UniqueName: \"kubernetes.io/projected/a73c6721-8084-4892-a0ff-9e30a7306dd0-kube-api-access-l9wzq\") pod \"payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") " pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.652069 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.652045    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/027af5a6-41ae-4dcd-813f-47dfc3df064b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.652155 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.652116    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73c6721-8084-4892-a0ff-9e30a7306dd0-plugins-config-volume\") pod \"payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") " pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.662276 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.662247    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9wzq\" (UniqueName: \"kubernetes.io/projected/a73c6721-8084-4892-a0ff-9e30a7306dd0-kube-api-access-l9wzq\") pod \"payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") " pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.662922 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.662890    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fnms\" (UniqueName: \"kubernetes.io/projected/027af5a6-41ae-4dcd-813f-47dfc3df064b-kube-api-access-4fnms\") pod \"payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.712941 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.712894    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:07.787656 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.787625    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:07.860556 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.860518    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"]
Jul 29 21:54:07.863335 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:54:07.862800    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027af5a6_41ae_4dcd_813f_47dfc3df064b.slice/crio-64e201d4ccfb7e0fb7c730cd633178ee4a2a04393a4a8353e7db660324a82dbb WatchSource:0}: Error finding container 64e201d4ccfb7e0fb7c730cd633178ee4a2a04393a4a8353e7db660324a82dbb: Status 404 returned error can't find the container with id 64e201d4ccfb7e0fb7c730cd633178ee4a2a04393a4a8353e7db660324a82dbb
Jul 29 21:54:07.938196 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:07.938158    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"]
Jul 29 21:54:07.940279 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:54:07.940248    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73c6721_8084_4892_a0ff_9e30a7306dd0.slice/crio-18a3193083acd28ec2bc5803ea023ef8db5414ac0d8388f19093ee481d157f45 WatchSource:0}: Error finding container 18a3193083acd28ec2bc5803ea023ef8db5414ac0d8388f19093ee481d157f45: Status 404 returned error can't find the container with id 18a3193083acd28ec2bc5803ea023ef8db5414ac0d8388f19093ee481d157f45
Jul 29 21:54:08.538798 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.538760    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9" event={"ID":"027af5a6-41ae-4dcd-813f-47dfc3df064b","Type":"ContainerStarted","Data":"1747c03e245e81a1c0080d7b7de0ad57698d8ba06ab35f9f61d855cb4a9195b2"}
Jul 29 21:54:08.539036 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.538836    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:08.539036 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.538864    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9" event={"ID":"027af5a6-41ae-4dcd-813f-47dfc3df064b","Type":"ContainerStarted","Data":"64e201d4ccfb7e0fb7c730cd633178ee4a2a04393a4a8353e7db660324a82dbb"}
Jul 29 21:54:08.540497 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.540462    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8" event={"ID":"a73c6721-8084-4892-a0ff-9e30a7306dd0","Type":"ContainerStarted","Data":"fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3"}
Jul 29 21:54:08.540623 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.540509    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8" event={"ID":"a73c6721-8084-4892-a0ff-9e30a7306dd0","Type":"ContainerStarted","Data":"18a3193083acd28ec2bc5803ea023ef8db5414ac0d8388f19093ee481d157f45"}
Jul 29 21:54:08.540623 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.540535    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:08.581571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.581515    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9" podStartSLOduration=1.581498023 podStartE2EDuration="1.581498023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:54:08.578242323 +0000 UTC m=+2259.795019786" watchObservedRunningTime="2026-07-29 21:54:08.581498023 +0000 UTC m=+2259.798275475"
Jul 29 21:54:08.620285 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:08.620229    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8" podStartSLOduration=1.6202136 podStartE2EDuration="1.6202136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:54:08.616525137 +0000 UTC m=+2259.833302603" watchObservedRunningTime="2026-07-29 21:54:08.6202136 +0000 UTC m=+2259.836991119"
Jul 29 21:54:12.947854 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:12.947815    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:54:19.548386 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:19.548353    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:19.548937 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:19.548435    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:30.376414 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:30.376331    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:31.083320 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:31.083284    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:31.170467 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:31.170433    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:31.553539 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:31.553507    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:37.392899 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:37.392868    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:37.491935 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:37.491902    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:48.435850 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.435807    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:48.466227 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.466188    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"]
Jul 29 21:54:48.466496 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.466448    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8" podUID="a73c6721-8084-4892-a0ff-9e30a7306dd0" containerName="payload-processing" containerID="cri-o://fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3" gracePeriod=30
Jul 29 21:54:48.473788 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.473751    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"]
Jul 29 21:54:48.474150 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.474121    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9" podUID="027af5a6-41ae-4dcd-813f-47dfc3df064b" containerName="payload-pre-processing" containerID="cri-o://1747c03e245e81a1c0080d7b7de0ad57698d8ba06ab35f9f61d855cb4a9195b2" gracePeriod=30
Jul 29 21:54:48.535682 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.535631    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:48.632438 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.632397    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:48.703126 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.703030    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:48.726384 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.726356    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:48.763508 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.763471    2572 generic.go:356] "Generic (PLEG): container finished" podID="027af5a6-41ae-4dcd-813f-47dfc3df064b" containerID="1747c03e245e81a1c0080d7b7de0ad57698d8ba06ab35f9f61d855cb4a9195b2" exitCode=0
Jul 29 21:54:48.763508 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.763492    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9" event={"ID":"027af5a6-41ae-4dcd-813f-47dfc3df064b","Type":"ContainerDied","Data":"1747c03e245e81a1c0080d7b7de0ad57698d8ba06ab35f9f61d855cb4a9195b2"}
Jul 29 21:54:48.763759 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.763547    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9" event={"ID":"027af5a6-41ae-4dcd-813f-47dfc3df064b","Type":"ContainerDied","Data":"64e201d4ccfb7e0fb7c730cd633178ee4a2a04393a4a8353e7db660324a82dbb"}
Jul 29 21:54:48.763759 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.763571    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64e201d4ccfb7e0fb7c730cd633178ee4a2a04393a4a8353e7db660324a82dbb"
Jul 29 21:54:48.765392 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.765365    2572 generic.go:356] "Generic (PLEG): container finished" podID="a73c6721-8084-4892-a0ff-9e30a7306dd0" containerID="fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3" exitCode=0
Jul 29 21:54:48.765568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.765428    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"
Jul 29 21:54:48.765568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.765455    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8" event={"ID":"a73c6721-8084-4892-a0ff-9e30a7306dd0","Type":"ContainerDied","Data":"fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3"}
Jul 29 21:54:48.765568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.765503    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8" event={"ID":"a73c6721-8084-4892-a0ff-9e30a7306dd0","Type":"ContainerDied","Data":"18a3193083acd28ec2bc5803ea023ef8db5414ac0d8388f19093ee481d157f45"}
Jul 29 21:54:48.765568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.765535    2572 scope.go:117] "RemoveContainer" containerID="fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3"
Jul 29 21:54:48.767300 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.767274    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:48.776210 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.776188    2572 scope.go:117] "RemoveContainer" containerID="fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3"
Jul 29 21:54:48.776501 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:54:48.776480    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3\": container with ID starting with fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3 not found: ID does not exist" containerID="fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3"
Jul 29 21:54:48.776589 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.776513    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3"} err="failed to get container status \"fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3\": rpc error: code = NotFound desc = could not find container \"fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3\": container with ID starting with fa3c1e4b3a93efe3bf5e2509d8f1a50e9277ea7ad88ea1a1f28c2857ba23f3c3 not found: ID does not exist"
Jul 29 21:54:48.821583 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.821558    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73c6721-8084-4892-a0ff-9e30a7306dd0-plugins-config-volume\") pod \"a73c6721-8084-4892-a0ff-9e30a7306dd0\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") "
Jul 29 21:54:48.821721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.821604    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/027af5a6-41ae-4dcd-813f-47dfc3df064b-plugins-config-volume\") pod \"027af5a6-41ae-4dcd-813f-47dfc3df064b\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") "
Jul 29 21:54:48.821721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.821627    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fnms\" (UniqueName: \"kubernetes.io/projected/027af5a6-41ae-4dcd-813f-47dfc3df064b-kube-api-access-4fnms\") pod \"027af5a6-41ae-4dcd-813f-47dfc3df064b\" (UID: \"027af5a6-41ae-4dcd-813f-47dfc3df064b\") "
Jul 29 21:54:48.821721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.821684    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9wzq\" (UniqueName: \"kubernetes.io/projected/a73c6721-8084-4892-a0ff-9e30a7306dd0-kube-api-access-l9wzq\") pod \"a73c6721-8084-4892-a0ff-9e30a7306dd0\" (UID: \"a73c6721-8084-4892-a0ff-9e30a7306dd0\") "
Jul 29 21:54:48.822006 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.821961    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73c6721-8084-4892-a0ff-9e30a7306dd0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a73c6721-8084-4892-a0ff-9e30a7306dd0" (UID: "a73c6721-8084-4892-a0ff-9e30a7306dd0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:54:48.822127 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.822010    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/027af5a6-41ae-4dcd-813f-47dfc3df064b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "027af5a6-41ae-4dcd-813f-47dfc3df064b" (UID: "027af5a6-41ae-4dcd-813f-47dfc3df064b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:54:48.823769 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.823745    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73c6721-8084-4892-a0ff-9e30a7306dd0-kube-api-access-l9wzq" (OuterVolumeSpecName: "kube-api-access-l9wzq") pod "a73c6721-8084-4892-a0ff-9e30a7306dd0" (UID: "a73c6721-8084-4892-a0ff-9e30a7306dd0"). InnerVolumeSpecName "kube-api-access-l9wzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:54:48.823895 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.823874    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/027af5a6-41ae-4dcd-813f-47dfc3df064b-kube-api-access-4fnms" (OuterVolumeSpecName: "kube-api-access-4fnms") pod "027af5a6-41ae-4dcd-813f-47dfc3df064b" (UID: "027af5a6-41ae-4dcd-813f-47dfc3df064b"). InnerVolumeSpecName "kube-api-access-4fnms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:54:48.922597 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.922552    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9wzq\" (UniqueName: \"kubernetes.io/projected/a73c6721-8084-4892-a0ff-9e30a7306dd0-kube-api-access-l9wzq\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:54:48.922597 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.922590    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a73c6721-8084-4892-a0ff-9e30a7306dd0-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:54:48.922597 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.922603    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/027af5a6-41ae-4dcd-813f-47dfc3df064b-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:54:48.922849 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:48.922616    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fnms\" (UniqueName: \"kubernetes.io/projected/027af5a6-41ae-4dcd-813f-47dfc3df064b-kube-api-access-4fnms\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:54:49.022091 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.021995    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:49.089506 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.089472    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"]
Jul 29 21:54:49.093894 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.093868    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-d5dee3c8-6d466cfc74-ltxh8"]
Jul 29 21:54:49.305936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.305860    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73c6721-8084-4892-a0ff-9e30a7306dd0" path="/var/lib/kubelet/pods/a73c6721-8084-4892-a0ff-9e30a7306dd0/volumes"
Jul 29 21:54:49.772571 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.772542    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"
Jul 29 21:54:49.811767 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.811729    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"]
Jul 29 21:54:49.817132 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:49.817100    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-d5dee3c8-57d759b696-56ff9"]
Jul 29 21:54:51.311896 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:51.311860    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="027af5a6-41ae-4dcd-813f-47dfc3df064b" path="/var/lib/kubelet/pods/027af5a6-41ae-4dcd-813f-47dfc3df064b/volumes"
Jul 29 21:54:51.335887 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:51.335855    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:54:55.002842 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.002801    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:55.106392 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.106346    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:55.268003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.267897    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:55.510618 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.510583    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:55.703897 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.703853    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:55.896618 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.896585    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:54:55.995619 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:54:55.995525    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:01.789131 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:01.789095    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:01.905585 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:01.905551    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:02.074580 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:02.074444    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:02.293080 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:02.293033    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:08.544147 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:08.544104    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:08.620652 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:08.620618    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:26.227081 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:26.226992    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:26.327076 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:26.327039    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:55:28.734885 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:28.734849    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:55:37.042288 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.042250    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"]
Jul 29 21:55:37.042777 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.042459    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56" podUID="07d33eee-10b0-4d5a-bb14-6e6736748bf1" containerName="manager" containerID="cri-o://676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca" gracePeriod=10
Jul 29 21:55:37.611464 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.611436    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:55:37.663578 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.663485    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07d33eee-10b0-4d5a-bb14-6e6736748bf1-extensions-socket-volume\") pod \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") "
Jul 29 21:55:37.663758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.663608    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m57v6\" (UniqueName: \"kubernetes.io/projected/07d33eee-10b0-4d5a-bb14-6e6736748bf1-kube-api-access-m57v6\") pod \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\" (UID: \"07d33eee-10b0-4d5a-bb14-6e6736748bf1\") "
Jul 29 21:55:37.663931 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.663905    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d33eee-10b0-4d5a-bb14-6e6736748bf1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "07d33eee-10b0-4d5a-bb14-6e6736748bf1" (UID: "07d33eee-10b0-4d5a-bb14-6e6736748bf1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:55:37.665816 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.665780    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d33eee-10b0-4d5a-bb14-6e6736748bf1-kube-api-access-m57v6" (OuterVolumeSpecName: "kube-api-access-m57v6") pod "07d33eee-10b0-4d5a-bb14-6e6736748bf1" (UID: "07d33eee-10b0-4d5a-bb14-6e6736748bf1"). InnerVolumeSpecName "kube-api-access-m57v6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:55:37.764449 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.764407    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m57v6\" (UniqueName: \"kubernetes.io/projected/07d33eee-10b0-4d5a-bb14-6e6736748bf1-kube-api-access-m57v6\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:55:37.764449 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:37.764443    2572 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07d33eee-10b0-4d5a-bb14-6e6736748bf1-extensions-socket-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:55:38.037136 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.037097    2572 generic.go:356] "Generic (PLEG): container finished" podID="07d33eee-10b0-4d5a-bb14-6e6736748bf1" containerID="676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca" exitCode=0
Jul 29 21:55:38.037359 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.037155    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"
Jul 29 21:55:38.037359 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.037199    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56" event={"ID":"07d33eee-10b0-4d5a-bb14-6e6736748bf1","Type":"ContainerDied","Data":"676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca"}
Jul 29 21:55:38.037359 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.037294    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56" event={"ID":"07d33eee-10b0-4d5a-bb14-6e6736748bf1","Type":"ContainerDied","Data":"6878a886c6858317a093e459a36e5156cfda9ea7609892062f89d58e3a581b9a"}
Jul 29 21:55:38.037359 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.037327    2572 scope.go:117] "RemoveContainer" containerID="676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca"
Jul 29 21:55:38.047930 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.047762    2572 scope.go:117] "RemoveContainer" containerID="676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca"
Jul 29 21:55:38.048136 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:55:38.048021    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca\": container with ID starting with 676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca not found: ID does not exist" containerID="676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca"
Jul 29 21:55:38.048136 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.048047    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca"} err="failed to get container status \"676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca\": rpc error: code = NotFound desc = could not find container \"676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca\": container with ID starting with 676b930fc788f422d1bf3cb55a5fe0b7ed6c793cd7eaf62186c606e02c459dca not found: ID does not exist"
Jul 29 21:55:38.062529 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.062506    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"]
Jul 29 21:55:38.072454 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:38.072428    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mjp56"]
Jul 29 21:55:39.304149 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:55:39.304111    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07d33eee-10b0-4d5a-bb14-6e6736748bf1" path="/var/lib/kubelet/pods/07d33eee-10b0-4d5a-bb14-6e6736748bf1/volumes"
Jul 29 21:56:38.302456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:38.302415    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:56:38.401246 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:38.401161    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:56:52.673794 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.673695    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"]
Jul 29 21:56:52.674241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674094    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07d33eee-10b0-4d5a-bb14-6e6736748bf1" containerName="manager"
Jul 29 21:56:52.674241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674104    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="07d33eee-10b0-4d5a-bb14-6e6736748bf1" containerName="manager"
Jul 29 21:56:52.674241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674114    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="027af5a6-41ae-4dcd-813f-47dfc3df064b" containerName="payload-pre-processing"
Jul 29 21:56:52.674241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674120    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="027af5a6-41ae-4dcd-813f-47dfc3df064b" containerName="payload-pre-processing"
Jul 29 21:56:52.674241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674143    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73c6721-8084-4892-a0ff-9e30a7306dd0" containerName="payload-processing"
Jul 29 21:56:52.674241 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674152    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73c6721-8084-4892-a0ff-9e30a7306dd0" containerName="payload-processing"
Jul 29 21:56:52.674486 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674250    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="07d33eee-10b0-4d5a-bb14-6e6736748bf1" containerName="manager"
Jul 29 21:56:52.674486 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674267    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a73c6721-8084-4892-a0ff-9e30a7306dd0" containerName="payload-processing"
Jul 29 21:56:52.674486 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.674277    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="027af5a6-41ae-4dcd-813f-47dfc3df064b" containerName="payload-pre-processing"
Jul 29 21:56:52.677602 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.677579    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.680306 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.680281    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nm827\""
Jul 29 21:56:52.691482 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.691449    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"]
Jul 29 21:56:52.766546 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.766502    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5h8s\" (UniqueName: \"kubernetes.io/projected/9c1c83f8-454c-4820-8da9-f5f3fd598708-kube-api-access-q5h8s\") pod \"kuadrant-operator-controller-manager-86878c769f-rgsh4\" (UID: \"9c1c83f8-454c-4820-8da9-f5f3fd598708\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.766546 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.766546    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1c83f8-454c-4820-8da9-f5f3fd598708-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rgsh4\" (UID: \"9c1c83f8-454c-4820-8da9-f5f3fd598708\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.867566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.867523    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5h8s\" (UniqueName: \"kubernetes.io/projected/9c1c83f8-454c-4820-8da9-f5f3fd598708-kube-api-access-q5h8s\") pod \"kuadrant-operator-controller-manager-86878c769f-rgsh4\" (UID: \"9c1c83f8-454c-4820-8da9-f5f3fd598708\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.867566 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.867568    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1c83f8-454c-4820-8da9-f5f3fd598708-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rgsh4\" (UID: \"9c1c83f8-454c-4820-8da9-f5f3fd598708\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.867930 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.867909    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9c1c83f8-454c-4820-8da9-f5f3fd598708-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rgsh4\" (UID: \"9c1c83f8-454c-4820-8da9-f5f3fd598708\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.879427 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.879397    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5h8s\" (UniqueName: \"kubernetes.io/projected/9c1c83f8-454c-4820-8da9-f5f3fd598708-kube-api-access-q5h8s\") pod \"kuadrant-operator-controller-manager-86878c769f-rgsh4\" (UID: \"9c1c83f8-454c-4820-8da9-f5f3fd598708\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:52.988255 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:52.988206    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:53.143722 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:53.143650    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"]
Jul 29 21:56:53.146540 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:56:53.146504    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c1c83f8_454c_4820_8da9_f5f3fd598708.slice/crio-cf923fa8a6499055ec68d69a7746c0f09f4d5a241811ab9ded275cab71704924 WatchSource:0}: Error finding container cf923fa8a6499055ec68d69a7746c0f09f4d5a241811ab9ded275cab71704924: Status 404 returned error can't find the container with id cf923fa8a6499055ec68d69a7746c0f09f4d5a241811ab9ded275cab71704924
Jul 29 21:56:53.440373 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:53.440335    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4" event={"ID":"9c1c83f8-454c-4820-8da9-f5f3fd598708","Type":"ContainerStarted","Data":"0af94cfbc8fd3621e02d39f49ba267eeca413ebf3191dc3b8cd69436508802fd"}
Jul 29 21:56:53.440568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:53.440395    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4" event={"ID":"9c1c83f8-454c-4820-8da9-f5f3fd598708","Type":"ContainerStarted","Data":"cf923fa8a6499055ec68d69a7746c0f09f4d5a241811ab9ded275cab71704924"}
Jul 29 21:56:53.440568 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:53.440417    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:56:53.460724 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:56:53.460670    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4" podStartSLOduration=1.460652938 podStartE2EDuration="1.460652938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:56:53.459206296 +0000 UTC m=+2424.675983759" watchObservedRunningTime="2026-07-29 21:56:53.460652938 +0000 UTC m=+2424.677430400"
Jul 29 21:57:04.447615 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:04.447579    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rgsh4"
Jul 29 21:57:10.991721 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:10.991645    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:11.318052 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:11.317924    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:11.867562 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:11.867524    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:14.120703 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:14.120654    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:14.208028 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:14.207993    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:15.069873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:15.069837    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:57:15.245426 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:15.245386    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:15.790325 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:15.790284    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:15.992089 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:15.992048    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:16.099321 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.099220    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"]
Jul 29 21:57:16.100370 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.100329    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:16.103417 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.103398    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.105758 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.105714    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-9bee5593\""
Jul 29 21:57:16.105902 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.105846    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-9bee5593-dockercfg-zqzkq\""
Jul 29 21:57:16.112892 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.112863    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"]
Jul 29 21:57:16.125573 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.125539    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"]
Jul 29 21:57:16.129197 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.129154    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.140028 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.140002    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"]
Jul 29 21:57:16.186003 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.185964    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rccck\" (UniqueName: \"kubernetes.io/projected/7381939c-9e92-46b9-ae02-96a777ce93ed-kube-api-access-rccck\") pod \"payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.186290 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.186263    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7381939c-9e92-46b9-ae02-96a777ce93ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.286963 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.286909    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rccck\" (UniqueName: \"kubernetes.io/projected/7381939c-9e92-46b9-ae02-96a777ce93ed-kube-api-access-rccck\") pod \"payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.287488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.287069    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-plugins-config-volume\") pod \"payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") " pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.287488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.287097    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rjn4\" (UniqueName: \"kubernetes.io/projected/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-kube-api-access-5rjn4\") pod \"payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") " pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.287488 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.287233    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7381939c-9e92-46b9-ae02-96a777ce93ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.287981 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.287963    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7381939c-9e92-46b9-ae02-96a777ce93ed-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.295350 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.295317    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rccck\" (UniqueName: \"kubernetes.io/projected/7381939c-9e92-46b9-ae02-96a777ce93ed-kube-api-access-rccck\") pod \"payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.388774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.388674    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-plugins-config-volume\") pod \"payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") " pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.388774 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.388715    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rjn4\" (UniqueName: \"kubernetes.io/projected/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-kube-api-access-5rjn4\") pod \"payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") " pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.389470 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.389431    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-plugins-config-volume\") pod \"payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") " pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.400680 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.400652    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rjn4\" (UniqueName: \"kubernetes.io/projected/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-kube-api-access-5rjn4\") pod \"payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") " pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.415485 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.415443    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:16.440339 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.440003    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:16.618048 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.617931    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"]
Jul 29 21:57:16.620291 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:57:16.620214    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7381939c_9e92_46b9_ae02_96a777ce93ed.slice/crio-18a936f3a3e8a1f406a920d94f88a4e7266eb3f3f010da322b1da77b84d3f287 WatchSource:0}: Error finding container 18a936f3a3e8a1f406a920d94f88a4e7266eb3f3f010da322b1da77b84d3f287: Status 404 returned error can't find the container with id 18a936f3a3e8a1f406a920d94f88a4e7266eb3f3f010da322b1da77b84d3f287
Jul 29 21:57:16.646497 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:16.646466    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"]
Jul 29 21:57:16.648614 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:57:16.648588    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d47341c_2ddf_4eae_ab17_b5fbc97e56e8.slice/crio-c54eb32b28d6c09de588c7d471c82ce3e91e0b52b6505ddd1bd8c7ef1bc0ec89 WatchSource:0}: Error finding container c54eb32b28d6c09de588c7d471c82ce3e91e0b52b6505ddd1bd8c7ef1bc0ec89: Status 404 returned error can't find the container with id c54eb32b28d6c09de588c7d471c82ce3e91e0b52b6505ddd1bd8c7ef1bc0ec89
Jul 29 21:57:17.583645 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.583603    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c" event={"ID":"7381939c-9e92-46b9-ae02-96a777ce93ed","Type":"ContainerStarted","Data":"e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc"}
Jul 29 21:57:17.584122 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.583810    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:17.584122 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.583851    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c" event={"ID":"7381939c-9e92-46b9-ae02-96a777ce93ed","Type":"ContainerStarted","Data":"18a936f3a3e8a1f406a920d94f88a4e7266eb3f3f010da322b1da77b84d3f287"}
Jul 29 21:57:17.585238 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.585210    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7" event={"ID":"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8","Type":"ContainerStarted","Data":"f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d"}
Jul 29 21:57:17.585378 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.585264    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:17.585378 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.585288    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7" event={"ID":"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8","Type":"ContainerStarted","Data":"c54eb32b28d6c09de588c7d471c82ce3e91e0b52b6505ddd1bd8c7ef1bc0ec89"}
Jul 29 21:57:17.601082 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.601038    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c" podStartSLOduration=1.6010275059999999 podStartE2EDuration="1.601027506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:57:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:57:17.60016079 +0000 UTC m=+2448.816938253" watchObservedRunningTime="2026-07-29 21:57:17.601027506 +0000 UTC m=+2448.817804967"
Jul 29 21:57:17.618639 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:17.618605    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7" podStartSLOduration=1.6185970589999998 podStartE2EDuration="1.618597059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:57:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:57:17.617944205 +0000 UTC m=+2448.834721667" watchObservedRunningTime="2026-07-29 21:57:17.618597059 +0000 UTC m=+2448.835374520"
Jul 29 21:57:27.733581 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:27.733538    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:57:28.593784 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:28.593755    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:57:28.593960 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:28.593823    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:57:39.124570 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:39.124535    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:42.206836 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.206798    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"]
Jul 29 21:57:42.210479 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.210455    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.213094 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.213071    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9bee5593\"/\"openshift-service-ca.crt\""
Jul 29 21:57:42.213283 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.213080    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9bee5593\"/\"kube-root-ca.crt\""
Jul 29 21:57:42.213283 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.213259    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9bee5593\"/\"default-dockercfg-w59r2\""
Jul 29 21:57:42.213896 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.213879    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9bee5593\"/\"e2e-lifecycle-model-9bee5593-kserve-self-signed-certs\""
Jul 29 21:57:42.225258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.225232    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"]
Jul 29 21:57:42.241239 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.241210    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:57:42.315615 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.315583    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-model-cache\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.315615 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.315616    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tls-certs\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.315879 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.315640    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-dshm\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.315879 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.315755    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tmp-dir\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.315879 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.315811    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-home\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.315879 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.315854    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf8lz\" (UniqueName: \"kubernetes.io/projected/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-kube-api-access-zf8lz\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417328 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-model-cache\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417328 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417331    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tls-certs\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417588 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417357    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-dshm\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417588 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417428    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tmp-dir\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417700 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417625    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-home\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417700 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417668    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf8lz\" (UniqueName: \"kubernetes.io/projected/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-kube-api-access-zf8lz\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417820 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417762    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-model-cache\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417877 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417819    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tmp-dir\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.417927 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.417906    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-home\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.419783 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.419752    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-dshm\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.420020 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.420002    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tls-certs\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.427307 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.427281    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf8lz\" (UniqueName: \"kubernetes.io/projected/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-kube-api-access-zf8lz\") pod \"e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") " pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.520658 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.520572    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:42.664517 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.664489    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"]
Jul 29 21:57:42.666990 ip-10-0-129-242 kubenswrapper[2572]: W0729 21:57:42.666959    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cc19b84_04a1_4611_9cf3_ec95a5c6f549.slice/crio-97745d913b1a2cee57395e589336a44310bf84a3e8f2e22baae76e4c3d09ccc4 WatchSource:0}: Error finding container 97745d913b1a2cee57395e589336a44310bf84a3e8f2e22baae76e4c3d09ccc4: Status 404 returned error can't find the container with id 97745d913b1a2cee57395e589336a44310bf84a3e8f2e22baae76e4c3d09ccc4
Jul 29 21:57:42.729246 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:42.729213    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl" event={"ID":"0cc19b84-04a1-4611-9cf3-ec95a5c6f549","Type":"ContainerStarted","Data":"97745d913b1a2cee57395e589336a44310bf84a3e8f2e22baae76e4c3d09ccc4"}
Jul 29 21:57:43.736978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:43.736939    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl" event={"ID":"0cc19b84-04a1-4611-9cf3-ec95a5c6f549","Type":"ContainerStarted","Data":"f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc"}
Jul 29 21:57:43.759257 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:43.759204    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl" podStartSLOduration=1.759187339 podStartE2EDuration="1.759187339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 21:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 21:57:43.757846466 +0000 UTC m=+2474.974623927" watchObservedRunningTime="2026-07-29 21:57:43.759187339 +0000 UTC m=+2474.975964794"
Jul 29 21:57:45.321299 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:45.321264    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:57:52.521123 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:52.521076    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:52.521123 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:52.521134    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:52.533885 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:52.533855    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:57:52.796936 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:57:52.796843    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:58:02.828893 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:02.828860    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:58:04.974731 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:04.974689    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:06.847476 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:06.847436    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:06.901794 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:06.901750    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:22.829673 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:22.829584    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:58:24.068633 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.068590    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:24.140648 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.140613    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"]
Jul 29 21:58:24.140901 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.140873    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7" podUID="6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" containerName="payload-processing" containerID="cri-o://f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d" gracePeriod=30
Jul 29 21:58:24.167238 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.167202    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"]
Jul 29 21:58:24.167451 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.167428    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c" podUID="7381939c-9e92-46b9-ae02-96a777ce93ed" containerName="payload-pre-processing" containerID="cri-o://e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc" gracePeriod=30
Jul 29 21:58:24.199010 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.198959    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:24.331011 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.330907    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:24.419795 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.419725    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:24.460687 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.460663    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:58:24.463428 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.463383    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:58:24.510093 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510058    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-plugins-config-volume\") pod \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") "
Jul 29 21:58:24.510315 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510197    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rjn4\" (UniqueName: \"kubernetes.io/projected/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-kube-api-access-5rjn4\") pod \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\" (UID: \"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8\") "
Jul 29 21:58:24.510315 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510250    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rccck\" (UniqueName: \"kubernetes.io/projected/7381939c-9e92-46b9-ae02-96a777ce93ed-kube-api-access-rccck\") pod \"7381939c-9e92-46b9-ae02-96a777ce93ed\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") "
Jul 29 21:58:24.510315 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510276    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7381939c-9e92-46b9-ae02-96a777ce93ed-plugins-config-volume\") pod \"7381939c-9e92-46b9-ae02-96a777ce93ed\" (UID: \"7381939c-9e92-46b9-ae02-96a777ce93ed\") "
Jul 29 21:58:24.510577 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510548    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" (UID: "6d47341c-2ddf-4eae-ab17-b5fbc97e56e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:58:24.510671 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510651    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:24.510733 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.510702    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7381939c-9e92-46b9-ae02-96a777ce93ed-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7381939c-9e92-46b9-ae02-96a777ce93ed" (UID: "7381939c-9e92-46b9-ae02-96a777ce93ed"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 21:58:24.512421 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.512393    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-kube-api-access-5rjn4" (OuterVolumeSpecName: "kube-api-access-5rjn4") pod "6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" (UID: "6d47341c-2ddf-4eae-ab17-b5fbc97e56e8"). InnerVolumeSpecName "kube-api-access-5rjn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:58:24.512899 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.512870    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7381939c-9e92-46b9-ae02-96a777ce93ed-kube-api-access-rccck" (OuterVolumeSpecName: "kube-api-access-rccck") pod "7381939c-9e92-46b9-ae02-96a777ce93ed" (UID: "7381939c-9e92-46b9-ae02-96a777ce93ed"). InnerVolumeSpecName "kube-api-access-rccck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:58:24.611915 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.611818    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rjn4\" (UniqueName: \"kubernetes.io/projected/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8-kube-api-access-5rjn4\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:24.611915 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.611851    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rccck\" (UniqueName: \"kubernetes.io/projected/7381939c-9e92-46b9-ae02-96a777ce93ed-kube-api-access-rccck\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:24.611915 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.611862    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7381939c-9e92-46b9-ae02-96a777ce93ed-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:24.975264 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.975220    2572 generic.go:356] "Generic (PLEG): container finished" podID="7381939c-9e92-46b9-ae02-96a777ce93ed" containerID="e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc" exitCode=0
Jul 29 21:58:24.975483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.975272    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"
Jul 29 21:58:24.975483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.975314    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c" event={"ID":"7381939c-9e92-46b9-ae02-96a777ce93ed","Type":"ContainerDied","Data":"e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc"}
Jul 29 21:58:24.975483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.975384    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c" event={"ID":"7381939c-9e92-46b9-ae02-96a777ce93ed","Type":"ContainerDied","Data":"18a936f3a3e8a1f406a920d94f88a4e7266eb3f3f010da322b1da77b84d3f287"}
Jul 29 21:58:24.975483 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.975400    2572 scope.go:117] "RemoveContainer" containerID="e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc"
Jul 29 21:58:24.977112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.977044    2572 generic.go:356] "Generic (PLEG): container finished" podID="6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" containerID="f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d" exitCode=0
Jul 29 21:58:24.977112 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.977105    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"
Jul 29 21:58:24.977332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.977141    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7" event={"ID":"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8","Type":"ContainerDied","Data":"f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d"}
Jul 29 21:58:24.977332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.977217    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7" event={"ID":"6d47341c-2ddf-4eae-ab17-b5fbc97e56e8","Type":"ContainerDied","Data":"c54eb32b28d6c09de588c7d471c82ce3e91e0b52b6505ddd1bd8c7ef1bc0ec89"}
Jul 29 21:58:24.987375 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.987353    2572 scope.go:117] "RemoveContainer" containerID="e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc"
Jul 29 21:58:24.987754 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:58:24.987725    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc\": container with ID starting with e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc not found: ID does not exist" containerID="e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc"
Jul 29 21:58:24.987853 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.987764    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc"} err="failed to get container status \"e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc\": rpc error: code = NotFound desc = could not find container \"e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc\": container with ID starting with e98259d6e0deefaeb03afa4c9f797033be14f069e43879c316450315437ea4dc not found: ID does not exist"
Jul 29 21:58:24.987853 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.987784    2572 scope.go:117] "RemoveContainer" containerID="f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d"
Jul 29 21:58:24.996601 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.996487    2572 scope.go:117] "RemoveContainer" containerID="f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d"
Jul 29 21:58:24.996888 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:58:24.996861    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d\": container with ID starting with f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d not found: ID does not exist" containerID="f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d"
Jul 29 21:58:24.996978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.996895    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d"} err="failed to get container status \"f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d\": rpc error: code = NotFound desc = could not find container \"f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d\": container with ID starting with f1c400c25c618eab5c96bf589708284f5454df088ea2db240fb32d2e7cc46a1d not found: ID does not exist"
Jul 29 21:58:24.999415 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:24.999394    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"]
Jul 29 21:58:25.003261 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:25.003207    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-9bee5593-7fb9dbf648-xhz2c"]
Jul 29 21:58:25.014465 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:25.014442    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"]
Jul 29 21:58:25.020383 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:25.020361    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-9bee5593-7bb75f8554-5zzg7"]
Jul 29 21:58:25.304850 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:25.304762    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" path="/var/lib/kubelet/pods/6d47341c-2ddf-4eae-ab17-b5fbc97e56e8/volumes"
Jul 29 21:58:25.305242 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:25.305219    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7381939c-9e92-46b9-ae02-96a777ce93ed" path="/var/lib/kubelet/pods/7381939c-9e92-46b9-ae02-96a777ce93ed/volumes"
Jul 29 21:58:27.712067 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:27.712024    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:31.944218 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:31.944180    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:58:32.732537 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:32.732494    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"]
Jul 29 21:58:32.802332 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:32.802285    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:39.278675 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:39.278627    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:58:47.754017 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:47.753964    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl" podUID="0cc19b84-04a1-4611-9cf3-ec95a5c6f549" containerName="main" containerID="cri-o://f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc" gracePeriod=45
Jul 29 21:58:47.781381 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:58:47.781350    2572 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cc19b84_04a1_4611_9cf3_ec95a5c6f549.slice/crio-f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cc19b84_04a1_4611_9cf3_ec95a5c6f549.slice/crio-conmon-f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc.scope\": RecentStats: unable to find data in memory cache]"
Jul 29 21:58:48.015456 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.015388    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:58:48.112274 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.112238    2572 generic.go:356] "Generic (PLEG): container finished" podID="0cc19b84-04a1-4611-9cf3-ec95a5c6f549" containerID="f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc" exitCode=0
Jul 29 21:58:48.112458 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.112283    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"
Jul 29 21:58:48.112458 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.112308    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl" event={"ID":"0cc19b84-04a1-4611-9cf3-ec95a5c6f549","Type":"ContainerDied","Data":"f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc"}
Jul 29 21:58:48.112458 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.112340    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl" event={"ID":"0cc19b84-04a1-4611-9cf3-ec95a5c6f549","Type":"ContainerDied","Data":"97745d913b1a2cee57395e589336a44310bf84a3e8f2e22baae76e4c3d09ccc4"}
Jul 29 21:58:48.112458 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.112357    2572 scope.go:117] "RemoveContainer" containerID="f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc"
Jul 29 21:58:48.121295 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.121273    2572 scope.go:117] "RemoveContainer" containerID="f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc"
Jul 29 21:58:48.121538 ip-10-0-129-242 kubenswrapper[2572]: E0729 21:58:48.121520    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc\": container with ID starting with f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc not found: ID does not exist" containerID="f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc"
Jul 29 21:58:48.121602 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.121547    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc"} err="failed to get container status \"f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc\": rpc error: code = NotFound desc = could not find container \"f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc\": container with ID starting with f8a8e628be29b5b35398200b164f8f42cf90085344206a28ce4dcd74707deedc not found: ID does not exist"
Jul 29 21:58:48.128873 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.128854    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-dshm\") pod \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") "
Jul 29 21:58:48.128940 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.128886    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-home\") pod \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") "
Jul 29 21:58:48.128978 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.128943    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-model-cache\") pod \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") "
Jul 29 21:58:48.129015 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.128989    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tmp-dir\") pod \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") "
Jul 29 21:58:48.129061 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129044    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zf8lz\" (UniqueName: \"kubernetes.io/projected/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-kube-api-access-zf8lz\") pod \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") "
Jul 29 21:58:48.129103 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129079    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tls-certs\") pod \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\" (UID: \"0cc19b84-04a1-4611-9cf3-ec95a5c6f549\") "
Jul 29 21:58:48.129283 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129251    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-model-cache" (OuterVolumeSpecName: "model-cache") pod "0cc19b84-04a1-4611-9cf3-ec95a5c6f549" (UID: "0cc19b84-04a1-4611-9cf3-ec95a5c6f549"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:48.129399 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129264    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-home" (OuterVolumeSpecName: "home") pod "0cc19b84-04a1-4611-9cf3-ec95a5c6f549" (UID: "0cc19b84-04a1-4611-9cf3-ec95a5c6f549"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:48.129464 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129391    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0cc19b84-04a1-4611-9cf3-ec95a5c6f549" (UID: "0cc19b84-04a1-4611-9cf3-ec95a5c6f549"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:48.129528 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129473    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:48.129528 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129489    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:48.129528 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.129501    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:48.131115 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.131092    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0cc19b84-04a1-4611-9cf3-ec95a5c6f549" (UID: "0cc19b84-04a1-4611-9cf3-ec95a5c6f549"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 21:58:48.131223 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.131141    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-dshm" (OuterVolumeSpecName: "dshm") pod "0cc19b84-04a1-4611-9cf3-ec95a5c6f549" (UID: "0cc19b84-04a1-4611-9cf3-ec95a5c6f549"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 21:58:48.131341 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.131291    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-kube-api-access-zf8lz" (OuterVolumeSpecName: "kube-api-access-zf8lz") pod "0cc19b84-04a1-4611-9cf3-ec95a5c6f549" (UID: "0cc19b84-04a1-4611-9cf3-ec95a5c6f549"). InnerVolumeSpecName "kube-api-access-zf8lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 21:58:48.230889 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.230852    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf8lz\" (UniqueName: \"kubernetes.io/projected/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-kube-api-access-zf8lz\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:48.230889 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.230885    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:48.230889 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.230894    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cc19b84-04a1-4611-9cf3-ec95a5c6f549-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 21:58:48.437636 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.437601    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"]
Jul 29 21:58:48.440230 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:48.440205    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9bee5593/e2e-lifecycle-model-9bee5593-kserve-d879bcf59-46kfl"]
Jul 29 21:58:49.304267 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:58:49.304232    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cc19b84-04a1-4611-9cf3-ec95a5c6f549" path="/var/lib/kubelet/pods/0cc19b84-04a1-4611-9cf3-ec95a5c6f549/volumes"
Jul 29 21:59:09.567427 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:09.567391    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:11.474159 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:11.474119    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:16.182400 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:16.182364    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:16.260258 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:16.260222    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:16.466761 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:16.466676    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:16.684871 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:16.684830    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:19.132294 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:19.132259    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:59:38.271189 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:38.271124    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:38.497701 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:38.497670    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:38.601473 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:38.601378    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:39.176649 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:39.176610    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:39.275543 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:39.275504    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:39.371441 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:39.371408    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:40.637281 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:40.637230    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:42.732876 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:42.732830    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:59:47.986430 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:47.986384    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:48.089580 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:48.089542    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:48.679850 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:48.679808    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:52.136053 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:52.135957    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 21:59:55.208097 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:55.208055    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:55.376414 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:55.376377    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:55.547278 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:55.547185    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 21:59:58.212760 ip-10-0-129-242 kubenswrapper[2572]: I0729 21:59:58.212719    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:05.991070 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:05.991036    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:06.321735 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:06.321646    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:27.221188 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:27.221138    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:29.609011 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:29.608980    2572 scope.go:117] "RemoveContainer" containerID="1747c03e245e81a1c0080d7b7de0ad57698d8ba06ab35f9f61d855cb4a9195b2"
Jul 29 22:00:34.548295 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:34.548256    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:34.649031 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:34.648997    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:34.961206 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:34.961139    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:35.318234 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:35.318139    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:35.939458 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:35.939421    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:38.367553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:38.367515    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:38.465532 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:38.465482    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:39.624367 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:39.624315    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:39.776199 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:39.776144    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:40.483331 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483277    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"]
Jul 29 22:00:40.483726 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483711    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" containerName="payload-processing"
Jul 29 22:00:40.483777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483727    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" containerName="payload-processing"
Jul 29 22:00:40.483777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483757    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc19b84-04a1-4611-9cf3-ec95a5c6f549" containerName="main"
Jul 29 22:00:40.483777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483763    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc19b84-04a1-4611-9cf3-ec95a5c6f549" containerName="main"
Jul 29 22:00:40.483777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483771    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7381939c-9e92-46b9-ae02-96a777ce93ed" containerName="payload-pre-processing"
Jul 29 22:00:40.483777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483776    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7381939c-9e92-46b9-ae02-96a777ce93ed" containerName="payload-pre-processing"
Jul 29 22:00:40.483930 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483833    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cc19b84-04a1-4611-9cf3-ec95a5c6f549" containerName="main"
Jul 29 22:00:40.483930 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483842    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d47341c-2ddf-4eae-ab17-b5fbc97e56e8" containerName="payload-processing"
Jul 29 22:00:40.483930 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.483848    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7381939c-9e92-46b9-ae02-96a777ce93ed" containerName="payload-pre-processing"
Jul 29 22:00:40.487001 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.486982    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"]
Jul 29 22:00:40.487111 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.487090    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.490539 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.490516    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.492707 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.492679    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-25c9e8\""
Jul 29 22:00:40.492911 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.492890    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-25c9e8-dockercfg-xlxdv\""
Jul 29 22:00:40.504516 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.504491    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"]
Jul 29 22:00:40.526667 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.526637    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"]
Jul 29 22:00:40.578699 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.578657    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9f685a-806c-4f2a-84a5-ae12602eaadf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.578918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.578712    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stzdw\" (UniqueName: \"kubernetes.io/projected/0c9f685a-806c-4f2a-84a5-ae12602eaadf-kube-api-access-stzdw\") pod \"payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.578918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.578838    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5gbt\" (UniqueName: \"kubernetes.io/projected/19a2790a-d727-4d1a-89d8-62312ebf532f-kube-api-access-h5gbt\") pod \"payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.579026 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.578939    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a2790a-d727-4d1a-89d8-62312ebf532f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.680126 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.680081    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a2790a-d727-4d1a-89d8-62312ebf532f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.680554 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.680139    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9f685a-806c-4f2a-84a5-ae12602eaadf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.680554 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.680240    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stzdw\" (UniqueName: \"kubernetes.io/projected/0c9f685a-806c-4f2a-84a5-ae12602eaadf-kube-api-access-stzdw\") pod \"payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.680554 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.680341    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5gbt\" (UniqueName: \"kubernetes.io/projected/19a2790a-d727-4d1a-89d8-62312ebf532f-kube-api-access-h5gbt\") pod \"payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.680920 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.680900    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9f685a-806c-4f2a-84a5-ae12602eaadf-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.681236 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.681215    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a2790a-d727-4d1a-89d8-62312ebf532f-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.689209 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.689188    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5gbt\" (UniqueName: \"kubernetes.io/projected/19a2790a-d727-4d1a-89d8-62312ebf532f-kube-api-access-h5gbt\") pod \"payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.689386 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.689371    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stzdw\" (UniqueName: \"kubernetes.io/projected/0c9f685a-806c-4f2a-84a5-ae12602eaadf-kube-api-access-stzdw\") pod \"payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") " pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.718064 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.718029    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:40.799508 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.799399    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:40.805832 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.805798    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:40.983465 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:40.983439    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"]
Jul 29 22:00:40.985381 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:00:40.985354    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c9f685a_806c_4f2a_84a5_ae12602eaadf.slice/crio-81507806c07acb725a3fbca4daa14bed2610fed2376c5c229f291b4d371ea5cf WatchSource:0}: Error finding container 81507806c07acb725a3fbca4daa14bed2610fed2376c5c229f291b4d371ea5cf: Status 404 returned error can't find the container with id 81507806c07acb725a3fbca4daa14bed2610fed2376c5c229f291b4d371ea5cf
Jul 29 22:00:41.166613 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.166583    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"]
Jul 29 22:00:41.169308 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:00:41.169261    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a2790a_d727_4d1a_89d8_62312ebf532f.slice/crio-25194b7c7e8c9d6fb660552fe308885cd70ccfedaa9277360d71f4e18ac4c003 WatchSource:0}: Error finding container 25194b7c7e8c9d6fb660552fe308885cd70ccfedaa9277360d71f4e18ac4c003: Status 404 returned error can't find the container with id 25194b7c7e8c9d6fb660552fe308885cd70ccfedaa9277360d71f4e18ac4c003
Jul 29 22:00:41.339506 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.339457    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:00:41.744779 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.744745    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct" event={"ID":"19a2790a-d727-4d1a-89d8-62312ebf532f","Type":"ContainerStarted","Data":"efa34b844249f046fb0da1e7cc1e11c7137557d87f772d1e3fe3983dd9300185"}
Jul 29 22:00:41.745299 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.744800    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct" event={"ID":"19a2790a-d727-4d1a-89d8-62312ebf532f","Type":"ContainerStarted","Data":"25194b7c7e8c9d6fb660552fe308885cd70ccfedaa9277360d71f4e18ac4c003"}
Jul 29 22:00:41.745299 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.744857    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:41.746326 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.746307    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86" event={"ID":"0c9f685a-806c-4f2a-84a5-ae12602eaadf","Type":"ContainerStarted","Data":"9240b1aa1d51821ee1e99091bee2bb9997c5d741b4586c225ef61c61818798f4"}
Jul 29 22:00:41.746416 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.746349    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86" event={"ID":"0c9f685a-806c-4f2a-84a5-ae12602eaadf","Type":"ContainerStarted","Data":"81507806c07acb725a3fbca4daa14bed2610fed2376c5c229f291b4d371ea5cf"}
Jul 29 22:00:41.746416 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.746379    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:00:41.772445 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.772393    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct" podStartSLOduration=1.7723733149999998 podStartE2EDuration="1.772373315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:00:41.76978789 +0000 UTC m=+2652.986565353" watchObservedRunningTime="2026-07-29 22:00:41.772373315 +0000 UTC m=+2652.989150782"
Jul 29 22:00:41.792920 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:41.792871    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86" podStartSLOduration=1.7928535220000001 podStartE2EDuration="1.792853522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:00:41.789512786 +0000 UTC m=+2653.006290250" watchObservedRunningTime="2026-07-29 22:00:41.792853522 +0000 UTC m=+2653.009630984"
Jul 29 22:00:43.746521 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:43.746486    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:00:52.753829 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:52.753796    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:00:52.754328 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:00:52.753874    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:01:03.315956 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:03.315912    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:03.707748 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:03.707709    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:04.241025 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:04.240986    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:04.421301 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:04.421271    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:08.420691 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:08.420655    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:08.522467 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:08.522433    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:08.770259 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:08.770223    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:08.871311 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:08.871260    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:09.336513 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.336474    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:09.440345 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.440316    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:09.556069 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.556039    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"]
Jul 29 22:01:09.559599 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.559582    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.562631 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.562601    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c24a76\""
Jul 29 22:01:09.562802 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.562785    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c24a76-dockercfg-49v76\""
Jul 29 22:01:09.573440 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.573410    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"]
Jul 29 22:01:09.639937 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.639860    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"]
Jul 29 22:01:09.643473 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.643446    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.648229 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.648199    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.648346 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.648308    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnszn\" (UniqueName: \"kubernetes.io/projected/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-kube-api-access-hnszn\") pod \"payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.655352 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.655327    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"]
Jul 29 22:01:09.749623 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.749592    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnszn\" (UniqueName: \"kubernetes.io/projected/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-kube-api-access-hnszn\") pod \"payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.749807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.749641    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcfqd\" (UniqueName: \"kubernetes.io/projected/845f7505-b1b6-406d-9541-dad7407cd22a-kube-api-access-lcfqd\") pod \"payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.749807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.749684    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/845f7505-b1b6-406d-9541-dad7407cd22a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.749807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.749717    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.750420 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.750401    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.758674 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.758651    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnszn\" (UniqueName: \"kubernetes.io/projected/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-kube-api-access-hnszn\") pod \"payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.851026 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.850978    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/845f7505-b1b6-406d-9541-dad7407cd22a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.851255 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.851099    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcfqd\" (UniqueName: \"kubernetes.io/projected/845f7505-b1b6-406d-9541-dad7407cd22a-kube-api-access-lcfqd\") pod \"payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.851727 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.851704    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/845f7505-b1b6-406d-9541-dad7407cd22a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.861720 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.861688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcfqd\" (UniqueName: \"kubernetes.io/projected/845f7505-b1b6-406d-9541-dad7407cd22a-kube-api-access-lcfqd\") pod \"payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:09.871034 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.871004    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:09.954921 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:09.954887    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:10.024662 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.024634    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"]
Jul 29 22:01:10.028831 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:01:10.028804    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83ea18e_4ad2_49e0_a3e8_10d601f7db7e.slice/crio-31f8e0b48cd0c99efafb421ece7d83f3b37cf1c1e5110ab81a3d18f71f7ffc64 WatchSource:0}: Error finding container 31f8e0b48cd0c99efafb421ece7d83f3b37cf1c1e5110ab81a3d18f71f7ffc64: Status 404 returned error can't find the container with id 31f8e0b48cd0c99efafb421ece7d83f3b37cf1c1e5110ab81a3d18f71f7ffc64
Jul 29 22:01:10.111343 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:01:10.111317    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845f7505_b1b6_406d_9541_dad7407cd22a.slice/crio-65c0a1cde544240af3d3e7039fc8fa2333f4dc37e1af49016c3c4a4f9b826d24 WatchSource:0}: Error finding container 65c0a1cde544240af3d3e7039fc8fa2333f4dc37e1af49016c3c4a4f9b826d24: Status 404 returned error can't find the container with id 65c0a1cde544240af3d3e7039fc8fa2333f4dc37e1af49016c3c4a4f9b826d24
Jul 29 22:01:10.111461 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.111429    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"]
Jul 29 22:01:10.914359 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.914318    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7" event={"ID":"845f7505-b1b6-406d-9541-dad7407cd22a","Type":"ContainerStarted","Data":"f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628"}
Jul 29 22:01:10.914830 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.914382    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7" event={"ID":"845f7505-b1b6-406d-9541-dad7407cd22a","Type":"ContainerStarted","Data":"65c0a1cde544240af3d3e7039fc8fa2333f4dc37e1af49016c3c4a4f9b826d24"}
Jul 29 22:01:10.914830 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.914410    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:10.916001 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.915977    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d" event={"ID":"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e","Type":"ContainerStarted","Data":"2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000"}
Jul 29 22:01:10.916112 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.916020    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:10.916112 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.916036    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d" event={"ID":"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e","Type":"ContainerStarted","Data":"31f8e0b48cd0c99efafb421ece7d83f3b37cf1c1e5110ab81a3d18f71f7ffc64"}
Jul 29 22:01:10.934682 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.934631    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7" podStartSLOduration=1.934614353 podStartE2EDuration="1.934614353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:01:10.933214631 +0000 UTC m=+2682.149992092" watchObservedRunningTime="2026-07-29 22:01:10.934614353 +0000 UTC m=+2682.151391815"
Jul 29 22:01:10.950322 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:10.950276    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d" podStartSLOduration=1.95025615 podStartE2EDuration="1.95025615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:01:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:01:10.949522038 +0000 UTC m=+2682.166299510" watchObservedRunningTime="2026-07-29 22:01:10.95025615 +0000 UTC m=+2682.167033612"
Jul 29 22:01:15.644986 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:15.644946    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:01:21.923413 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:21.923333    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:01:21.923831 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:21.923567    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:01:32.629673 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:32.629626    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:45.516388 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.516351    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"]
Jul 29 22:01:45.520457 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.520434    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.523116 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.523090    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-25c9e8\"/\"openshift-service-ca.crt\""
Jul 29 22:01:45.523285 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.523114    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-25c9e8\"/\"default-dockercfg-fpbrm\""
Jul 29 22:01:45.524004 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.523984    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-25c9e8\"/\"kube-root-ca.crt\""
Jul 29 22:01:45.524117 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.524033    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-25c9e8\"/\"test-model-25c9e8-kserve-self-signed-certs\""
Jul 29 22:01:45.533156 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.533129    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"]
Jul 29 22:01:45.573768 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.573732    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-home\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.573959 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.573799    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc69v\" (UniqueName: \"kubernetes.io/projected/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-kube-api-access-cc69v\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.573959 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.573846    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tmp-dir\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.574097 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.573962    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tls-certs\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.574155 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.574118    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-dshm\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.574227 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.574157    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-model-cache\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.638833 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.638787    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:01:45.674920 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.674885    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tls-certs\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675107 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.674992    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-dshm\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675107 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675024    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-model-cache\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675107 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675056    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-home\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675107 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675102    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc69v\" (UniqueName: \"kubernetes.io/projected/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-kube-api-access-cc69v\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675347 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675145    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tmp-dir\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675542 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675516    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-home\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675664 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675576    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-model-cache\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.675731 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.675688    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tmp-dir\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.677281 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.677255    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-dshm\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.677544 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.677527    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tls-certs\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.683335 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.683315    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc69v\" (UniqueName: \"kubernetes.io/projected/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-kube-api-access-cc69v\") pod \"test-model-25c9e8-kserve-7ccc946797-dpncp\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") " pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.831700 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.831597    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:45.973314 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:45.973281    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"]
Jul 29 22:01:45.977009 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:01:45.976983    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9283dd4_f5cb_48d5_88eb_e9074ca8230a.slice/crio-d5c760092c059e8551427b1db0d67a3231154cc8fb614f35632d5ed355609e62 WatchSource:0}: Error finding container d5c760092c059e8551427b1db0d67a3231154cc8fb614f35632d5ed355609e62: Status 404 returned error can't find the container with id d5c760092c059e8551427b1db0d67a3231154cc8fb614f35632d5ed355609e62
Jul 29 22:01:46.107569 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:46.107485    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp" event={"ID":"b9283dd4-f5cb-48d5-88eb-e9074ca8230a","Type":"ContainerStarted","Data":"626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d"}
Jul 29 22:01:46.107569 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:46.107528    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp" event={"ID":"b9283dd4-f5cb-48d5-88eb-e9074ca8230a","Type":"ContainerStarted","Data":"d5c760092c059e8551427b1db0d67a3231154cc8fb614f35632d5ed355609e62"}
Jul 29 22:01:46.136680 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:46.136632    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp" podStartSLOduration=1.136617123 podStartE2EDuration="1.136617123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:01:46.135146454 +0000 UTC m=+2717.351923916" watchObservedRunningTime="2026-07-29 22:01:46.136617123 +0000 UTC m=+2717.353394584"
Jul 29 22:01:49.446347 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:49.446312    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:01:55.832428 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:55.832388    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:55.832428 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:55.832440    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:55.844983 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:55.844950    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:56.181579 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:56.181545    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:01:59.235992 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:01:59.235959    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:02:07.066337 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:07.066289    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:10.237402 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:10.237360    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:02:27.746254 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:27.746101    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:02:42.864523 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:42.864465    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:52.634408 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:52.634332    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"]
Jul 29 22:02:52.660314 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:52.660281    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:53.666041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:53.665970    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:02:59.642245 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:02:59.642205    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:03:04.312321 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.312277    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:04.374796 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.374760    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"]
Jul 29 22:03:04.375064 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.375031    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86" podUID="0c9f685a-806c-4f2a-84a5-ae12602eaadf" containerName="payload-processing" containerID="cri-o://9240b1aa1d51821ee1e99091bee2bb9997c5d741b4586c225ef61c61818798f4" gracePeriod=30
Jul 29 22:03:04.402119 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.402085    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"]
Jul 29 22:03:04.402388 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.402360    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct" podUID="19a2790a-d727-4d1a-89d8-62312ebf532f" containerName="payload-pre-processing" containerID="cri-o://efa34b844249f046fb0da1e7cc1e11c7137557d87f772d1e3fe3983dd9300185" gracePeriod=30
Jul 29 22:03:04.461720 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.461673    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:04.541601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.541547    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:04.564530 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.564441    2572 generic.go:356] "Generic (PLEG): container finished" podID="19a2790a-d727-4d1a-89d8-62312ebf532f" containerID="efa34b844249f046fb0da1e7cc1e11c7137557d87f772d1e3fe3983dd9300185" exitCode=0
Jul 29 22:03:04.565198 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.564740    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct" event={"ID":"19a2790a-d727-4d1a-89d8-62312ebf532f","Type":"ContainerDied","Data":"efa34b844249f046fb0da1e7cc1e11c7137557d87f772d1e3fe3983dd9300185"}
Jul 29 22:03:04.568603 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.568574    2572 generic.go:356] "Generic (PLEG): container finished" podID="0c9f685a-806c-4f2a-84a5-ae12602eaadf" containerID="9240b1aa1d51821ee1e99091bee2bb9997c5d741b4586c225ef61c61818798f4" exitCode=0
Jul 29 22:03:04.568836 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.568766    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86" event={"ID":"0c9f685a-806c-4f2a-84a5-ae12602eaadf","Type":"ContainerDied","Data":"9240b1aa1d51821ee1e99091bee2bb9997c5d741b4586c225ef61c61818798f4"}
Jul 29 22:03:04.628792 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.628753    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:04.697352 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.697327    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:03:04.700307 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.700283    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:03:04.829089 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.828990    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stzdw\" (UniqueName: \"kubernetes.io/projected/0c9f685a-806c-4f2a-84a5-ae12602eaadf-kube-api-access-stzdw\") pod \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") "
Jul 29 22:03:04.829089 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.829045    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a2790a-d727-4d1a-89d8-62312ebf532f-plugins-config-volume\") pod \"19a2790a-d727-4d1a-89d8-62312ebf532f\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") "
Jul 29 22:03:04.829375 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.829095    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9f685a-806c-4f2a-84a5-ae12602eaadf-plugins-config-volume\") pod \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\" (UID: \"0c9f685a-806c-4f2a-84a5-ae12602eaadf\") "
Jul 29 22:03:04.829375 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.829130    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5gbt\" (UniqueName: \"kubernetes.io/projected/19a2790a-d727-4d1a-89d8-62312ebf532f-kube-api-access-h5gbt\") pod \"19a2790a-d727-4d1a-89d8-62312ebf532f\" (UID: \"19a2790a-d727-4d1a-89d8-62312ebf532f\") "
Jul 29 22:03:04.829520 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.829493    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c9f685a-806c-4f2a-84a5-ae12602eaadf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c9f685a-806c-4f2a-84a5-ae12602eaadf" (UID: "0c9f685a-806c-4f2a-84a5-ae12602eaadf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:04.829592 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.829506    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19a2790a-d727-4d1a-89d8-62312ebf532f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "19a2790a-d727-4d1a-89d8-62312ebf532f" (UID: "19a2790a-d727-4d1a-89d8-62312ebf532f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:04.831281 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.831255    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9f685a-806c-4f2a-84a5-ae12602eaadf-kube-api-access-stzdw" (OuterVolumeSpecName: "kube-api-access-stzdw") pod "0c9f685a-806c-4f2a-84a5-ae12602eaadf" (UID: "0c9f685a-806c-4f2a-84a5-ae12602eaadf"). InnerVolumeSpecName "kube-api-access-stzdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:04.831401 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.831290    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a2790a-d727-4d1a-89d8-62312ebf532f-kube-api-access-h5gbt" (OuterVolumeSpecName: "kube-api-access-h5gbt") pod "19a2790a-d727-4d1a-89d8-62312ebf532f" (UID: "19a2790a-d727-4d1a-89d8-62312ebf532f"). InnerVolumeSpecName "kube-api-access-h5gbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:04.930964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.930925    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stzdw\" (UniqueName: \"kubernetes.io/projected/0c9f685a-806c-4f2a-84a5-ae12602eaadf-kube-api-access-stzdw\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:04.930964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.930957    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/19a2790a-d727-4d1a-89d8-62312ebf532f-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:04.930964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.930968    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c9f685a-806c-4f2a-84a5-ae12602eaadf-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:04.931247 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:04.930978    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5gbt\" (UniqueName: \"kubernetes.io/projected/19a2790a-d727-4d1a-89d8-62312ebf532f-kube-api-access-h5gbt\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:05.035513 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.035477    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:05.577618 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.577531    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"
Jul 29 22:03:05.577618 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.577532    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct" event={"ID":"19a2790a-d727-4d1a-89d8-62312ebf532f","Type":"ContainerDied","Data":"25194b7c7e8c9d6fb660552fe308885cd70ccfedaa9277360d71f4e18ac4c003"}
Jul 29 22:03:05.578148 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.577645    2572 scope.go:117] "RemoveContainer" containerID="efa34b844249f046fb0da1e7cc1e11c7137557d87f772d1e3fe3983dd9300185"
Jul 29 22:03:05.579057 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.578979    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"
Jul 29 22:03:05.579057 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.579029    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86" event={"ID":"0c9f685a-806c-4f2a-84a5-ae12602eaadf","Type":"ContainerDied","Data":"81507806c07acb725a3fbca4daa14bed2610fed2376c5c229f291b4d371ea5cf"}
Jul 29 22:03:05.587688 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.587670    2572 scope.go:117] "RemoveContainer" containerID="9240b1aa1d51821ee1e99091bee2bb9997c5d741b4586c225ef61c61818798f4"
Jul 29 22:03:05.602824 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.602792    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"]
Jul 29 22:03:05.610090 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.610056    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-25c9e8-5567949664-xg4ct"]
Jul 29 22:03:05.621783 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.621749    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"]
Jul 29 22:03:05.629087 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:05.629054    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-25c9e8-674d9585f4-6mh86"]
Jul 29 22:03:07.310134 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.310097    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c9f685a-806c-4f2a-84a5-ae12602eaadf" path="/var/lib/kubelet/pods/0c9f685a-806c-4f2a-84a5-ae12602eaadf/volumes"
Jul 29 22:03:07.310550 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.310499    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a2790a-d727-4d1a-89d8-62312ebf532f" path="/var/lib/kubelet/pods/19a2790a-d727-4d1a-89d8-62312ebf532f/volumes"
Jul 29 22:03:07.653409 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.653319    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp" podUID="b9283dd4-f5cb-48d5-88eb-e9074ca8230a" containerName="main" containerID="cri-o://626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d" gracePeriod=45
Jul 29 22:03:07.909574 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.909512    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:03:07.956935 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.956881    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-model-cache\") pod \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") "
Jul 29 22:03:07.957145 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.956974    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tmp-dir\") pod \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") "
Jul 29 22:03:07.957145 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.956999    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-dshm\") pod \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") "
Jul 29 22:03:07.957145 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957026    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-home\") pod \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") "
Jul 29 22:03:07.957145 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957064    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tls-certs\") pod \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") "
Jul 29 22:03:07.957427 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957151    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc69v\" (UniqueName: \"kubernetes.io/projected/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-kube-api-access-cc69v\") pod \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\" (UID: \"b9283dd4-f5cb-48d5-88eb-e9074ca8230a\") "
Jul 29 22:03:07.957427 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957194    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-model-cache" (OuterVolumeSpecName: "model-cache") pod "b9283dd4-f5cb-48d5-88eb-e9074ca8230a" (UID: "b9283dd4-f5cb-48d5-88eb-e9074ca8230a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:03:07.957427 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957279    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b9283dd4-f5cb-48d5-88eb-e9074ca8230a" (UID: "b9283dd4-f5cb-48d5-88eb-e9074ca8230a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:03:07.957427 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957354    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-home" (OuterVolumeSpecName: "home") pod "b9283dd4-f5cb-48d5-88eb-e9074ca8230a" (UID: "b9283dd4-f5cb-48d5-88eb-e9074ca8230a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:03:07.957643 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957473    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:07.957643 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957488    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:07.957643 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.957505    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:07.959343 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.959312    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b9283dd4-f5cb-48d5-88eb-e9074ca8230a" (UID: "b9283dd4-f5cb-48d5-88eb-e9074ca8230a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:03:07.959463 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.959363    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-kube-api-access-cc69v" (OuterVolumeSpecName: "kube-api-access-cc69v") pod "b9283dd4-f5cb-48d5-88eb-e9074ca8230a" (UID: "b9283dd4-f5cb-48d5-88eb-e9074ca8230a"). InnerVolumeSpecName "kube-api-access-cc69v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:07.959463 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:07.959366    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-dshm" (OuterVolumeSpecName: "dshm") pod "b9283dd4-f5cb-48d5-88eb-e9074ca8230a" (UID: "b9283dd4-f5cb-48d5-88eb-e9074ca8230a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:03:08.058871 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.058835    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:08.058871 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.058865    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:08.058871 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.058876    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc69v\" (UniqueName: \"kubernetes.io/projected/b9283dd4-f5cb-48d5-88eb-e9074ca8230a-kube-api-access-cc69v\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:08.599610 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.599569    2572 generic.go:356] "Generic (PLEG): container finished" podID="b9283dd4-f5cb-48d5-88eb-e9074ca8230a" containerID="626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d" exitCode=0
Jul 29 22:03:08.600024 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.599618    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"
Jul 29 22:03:08.600024 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.599657    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp" event={"ID":"b9283dd4-f5cb-48d5-88eb-e9074ca8230a","Type":"ContainerDied","Data":"626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d"}
Jul 29 22:03:08.600024 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.599708    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp" event={"ID":"b9283dd4-f5cb-48d5-88eb-e9074ca8230a","Type":"ContainerDied","Data":"d5c760092c059e8551427b1db0d67a3231154cc8fb614f35632d5ed355609e62"}
Jul 29 22:03:08.600024 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.599730    2572 scope.go:117] "RemoveContainer" containerID="626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d"
Jul 29 22:03:08.610288 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.610269    2572 scope.go:117] "RemoveContainer" containerID="626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d"
Jul 29 22:03:08.610552 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:03:08.610532    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d\": container with ID starting with 626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d not found: ID does not exist" containerID="626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d"
Jul 29 22:03:08.610617 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.610561    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d"} err="failed to get container status \"626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d\": rpc error: code = NotFound desc = could not find container \"626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d\": container with ID starting with 626e14c7ee59a1ffc64711147f4d442f149b221a2cf4960273c1ea8ef1716c6d not found: ID does not exist"
Jul 29 22:03:08.623730 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.623704    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"]
Jul 29 22:03:08.629861 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:08.629836    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-25c9e8/test-model-25c9e8-kserve-7ccc946797-dpncp"]
Jul 29 22:03:09.304461 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:09.304422    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9283dd4-f5cb-48d5-88eb-e9074ca8230a" path="/var/lib/kubelet/pods/b9283dd4-f5cb-48d5-88eb-e9074ca8230a/volumes"
Jul 29 22:03:09.460699 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:09.460660    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:03:10.020666 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:10.020627    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:10.104185 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:10.104134    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:12.469688 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:12.469642    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:12.571000 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:12.570962    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:22.869786 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:22.869709    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:22.895887 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:22.895854    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"]
Jul 29 22:03:22.896120 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:22.896093    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7" podUID="845f7505-b1b6-406d-9541-dad7407cd22a" containerName="payload-processing" containerID="cri-o://f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628" gracePeriod=30
Jul 29 22:03:22.915014 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:22.914990    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"]
Jul 29 22:03:22.915231 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:22.915209    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d" podUID="e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" containerName="payload-pre-processing" containerID="cri-o://2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000" gracePeriod=30
Jul 29 22:03:22.974084 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:22.974056    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:23.072238 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.072149    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:23.166799 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.166772    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:03:23.176021 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.175992    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:23.199271 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.199240    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:03:23.304612 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.304582    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-plugins-config-volume\") pod \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") "
Jul 29 22:03:23.304809 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.304623    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcfqd\" (UniqueName: \"kubernetes.io/projected/845f7505-b1b6-406d-9541-dad7407cd22a-kube-api-access-lcfqd\") pod \"845f7505-b1b6-406d-9541-dad7407cd22a\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") "
Jul 29 22:03:23.304809 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.304655    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/845f7505-b1b6-406d-9541-dad7407cd22a-plugins-config-volume\") pod \"845f7505-b1b6-406d-9541-dad7407cd22a\" (UID: \"845f7505-b1b6-406d-9541-dad7407cd22a\") "
Jul 29 22:03:23.304809 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.304746    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnszn\" (UniqueName: \"kubernetes.io/projected/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-kube-api-access-hnszn\") pod \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\" (UID: \"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e\") "
Jul 29 22:03:23.305012 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.304971    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" (UID: "e83ea18e-4ad2-49e0-a3e8-10d601f7db7e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:23.305159 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.305137    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/845f7505-b1b6-406d-9541-dad7407cd22a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "845f7505-b1b6-406d-9541-dad7407cd22a" (UID: "845f7505-b1b6-406d-9541-dad7407cd22a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:03:23.306780 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.306755    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845f7505-b1b6-406d-9541-dad7407cd22a-kube-api-access-lcfqd" (OuterVolumeSpecName: "kube-api-access-lcfqd") pod "845f7505-b1b6-406d-9541-dad7407cd22a" (UID: "845f7505-b1b6-406d-9541-dad7407cd22a"). InnerVolumeSpecName "kube-api-access-lcfqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:23.306903 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.306878    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-kube-api-access-hnszn" (OuterVolumeSpecName: "kube-api-access-hnszn") pod "e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" (UID: "e83ea18e-4ad2-49e0-a3e8-10d601f7db7e"). InnerVolumeSpecName "kube-api-access-hnszn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:03:23.406475 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.406362    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnszn\" (UniqueName: \"kubernetes.io/projected/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-kube-api-access-hnszn\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:23.406475 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.406402    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:23.406475 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.406417    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcfqd\" (UniqueName: \"kubernetes.io/projected/845f7505-b1b6-406d-9541-dad7407cd22a-kube-api-access-lcfqd\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:23.406475 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.406431    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/845f7505-b1b6-406d-9541-dad7407cd22a-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:03:23.628519 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.628486    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:23.689231 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.689198    2572 generic.go:356] "Generic (PLEG): container finished" podID="845f7505-b1b6-406d-9541-dad7407cd22a" containerID="f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628" exitCode=0
Jul 29 22:03:23.689414 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.689242    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"
Jul 29 22:03:23.689414 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.689263    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7" event={"ID":"845f7505-b1b6-406d-9541-dad7407cd22a","Type":"ContainerDied","Data":"f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628"}
Jul 29 22:03:23.689414 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.689321    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7" event={"ID":"845f7505-b1b6-406d-9541-dad7407cd22a","Type":"ContainerDied","Data":"65c0a1cde544240af3d3e7039fc8fa2333f4dc37e1af49016c3c4a4f9b826d24"}
Jul 29 22:03:23.689414 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.689352    2572 scope.go:117] "RemoveContainer" containerID="f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628"
Jul 29 22:03:23.690759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.690734    2572 generic.go:356] "Generic (PLEG): container finished" podID="e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" containerID="2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000" exitCode=0
Jul 29 22:03:23.690882 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.690790    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d" event={"ID":"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e","Type":"ContainerDied","Data":"2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000"}
Jul 29 22:03:23.690882 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.690798    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"
Jul 29 22:03:23.690882 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.690826    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d" event={"ID":"e83ea18e-4ad2-49e0-a3e8-10d601f7db7e","Type":"ContainerDied","Data":"31f8e0b48cd0c99efafb421ece7d83f3b37cf1c1e5110ab81a3d18f71f7ffc64"}
Jul 29 22:03:23.701662 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.701637    2572 scope.go:117] "RemoveContainer" containerID="f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628"
Jul 29 22:03:23.702271 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:03:23.702150    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628\": container with ID starting with f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628 not found: ID does not exist" containerID="f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628"
Jul 29 22:03:23.702271 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.702194    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628"} err="failed to get container status \"f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628\": rpc error: code = NotFound desc = could not find container \"f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628\": container with ID starting with f0cbb163f6b568e455a7d3adb9c3f5d30eaae5737b2548713843fe194c9c3628 not found: ID does not exist"
Jul 29 22:03:23.702271 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.702213    2572 scope.go:117] "RemoveContainer" containerID="2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000"
Jul 29 22:03:23.710719 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.710701    2572 scope.go:117] "RemoveContainer" containerID="2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000"
Jul 29 22:03:23.710968 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:03:23.710947    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000\": container with ID starting with 2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000 not found: ID does not exist" containerID="2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000"
Jul 29 22:03:23.711025 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.710974    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000"} err="failed to get container status \"2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000\": rpc error: code = NotFound desc = could not find container \"2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000\": container with ID starting with 2f1ef896edca74220ae2258821be4fe4e8dd1017b4281dca7d501cb887b9d000 not found: ID does not exist"
Jul 29 22:03:23.720354 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.720327    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"]
Jul 29 22:03:23.725017 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.724991    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-c24a76-77dfdcb456-x649d"]
Jul 29 22:03:23.739596 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.739563    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"]
Jul 29 22:03:23.748349 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:23.748319    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c24a76-55d59dcf86-kkjd7"]
Jul 29 22:03:25.304411 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:25.304357    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845f7505-b1b6-406d-9541-dad7407cd22a" path="/var/lib/kubelet/pods/845f7505-b1b6-406d-9541-dad7407cd22a/volumes"
Jul 29 22:03:25.304918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:25.304902    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" path="/var/lib/kubelet/pods/e83ea18e-4ad2-49e0-a3e8-10d601f7db7e/volumes"
Jul 29 22:03:26.150369 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:26.150330    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:03:28.117053 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:28.117012    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:29.933475 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:29.933436    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:30.033871 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:30.033826    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:30.223380 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:30.223292    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:30.504618 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:30.504529    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:30.846757 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:30.846660    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:31.218836 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:31.218802    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:31.297812 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:31.297769    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:34.380463 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:34.380412    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:34.533207 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:34.533155    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:35.700087 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:35.700034    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:36.022873 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.022776    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:36.552534 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.552501    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:36.717271 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.717232    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:03:36.820131 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820042    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"]
Jul 29 22:03:36.820500 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820486    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845f7505-b1b6-406d-9541-dad7407cd22a" containerName="payload-processing"
Jul 29 22:03:36.820549 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820503    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="845f7505-b1b6-406d-9541-dad7407cd22a" containerName="payload-processing"
Jul 29 22:03:36.820549 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820513    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" containerName="payload-pre-processing"
Jul 29 22:03:36.820549 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820520    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" containerName="payload-pre-processing"
Jul 29 22:03:36.820549 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820528    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9283dd4-f5cb-48d5-88eb-e9074ca8230a" containerName="main"
Jul 29 22:03:36.820549 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820533    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9283dd4-f5cb-48d5-88eb-e9074ca8230a" containerName="main"
Jul 29 22:03:36.820549 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820549    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c9f685a-806c-4f2a-84a5-ae12602eaadf" containerName="payload-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820555    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c9f685a-806c-4f2a-84a5-ae12602eaadf" containerName="payload-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820567    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a2790a-d727-4d1a-89d8-62312ebf532f" containerName="payload-pre-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820573    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a2790a-d727-4d1a-89d8-62312ebf532f" containerName="payload-pre-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820635    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a2790a-d727-4d1a-89d8-62312ebf532f" containerName="payload-pre-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820644    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c9f685a-806c-4f2a-84a5-ae12602eaadf" containerName="payload-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820653    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83ea18e-4ad2-49e0-a3e8-10d601f7db7e" containerName="payload-pre-processing"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820659    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9283dd4-f5cb-48d5-88eb-e9074ca8230a" containerName="main"
Jul 29 22:03:36.820733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.820666    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="845f7505-b1b6-406d-9541-dad7407cd22a" containerName="payload-processing"
Jul 29 22:03:36.831627 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.831596    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:36.835874 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.835845    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c8e5f0-dockercfg-8j9b5\""
Jul 29 22:03:36.836057 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.835846    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c8e5f0\""
Jul 29 22:03:36.856655 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.856624    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"]
Jul 29 22:03:36.863669 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.863636    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"]
Jul 29 22:03:36.867325 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.867296    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:36.893294 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.893258    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"]
Jul 29 22:03:36.924977 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.924945    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp6ml\" (UniqueName: \"kubernetes.io/projected/94dcabcb-054d-42ef-bbe0-5d37823ab154-kube-api-access-rp6ml\") pod \"payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") " pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:36.925159 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.924993    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhbts\" (UniqueName: \"kubernetes.io/projected/85f8412b-bebe-4b58-85c7-e468c54466eb-kube-api-access-hhbts\") pod \"payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:36.925159 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.925057    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94dcabcb-054d-42ef-bbe0-5d37823ab154-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") " pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:36.925159 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:36.925098    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f8412b-bebe-4b58-85c7-e468c54466eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.026544 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.026502    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f8412b-bebe-4b58-85c7-e468c54466eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.026739 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.026573    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp6ml\" (UniqueName: \"kubernetes.io/projected/94dcabcb-054d-42ef-bbe0-5d37823ab154-kube-api-access-rp6ml\") pod \"payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") " pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:37.026739 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.026608    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhbts\" (UniqueName: \"kubernetes.io/projected/85f8412b-bebe-4b58-85c7-e468c54466eb-kube-api-access-hhbts\") pod \"payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.026739 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.026645    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94dcabcb-054d-42ef-bbe0-5d37823ab154-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") " pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:37.027314 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.027294    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94dcabcb-054d-42ef-bbe0-5d37823ab154-plugins-config-volume\") pod \"payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") " pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:37.027363 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.027293    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f8412b-bebe-4b58-85c7-e468c54466eb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.047862 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.047822    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp6ml\" (UniqueName: \"kubernetes.io/projected/94dcabcb-054d-42ef-bbe0-5d37823ab154-kube-api-access-rp6ml\") pod \"payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") " pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:37.049381 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.049355    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhbts\" (UniqueName: \"kubernetes.io/projected/85f8412b-bebe-4b58-85c7-e468c54466eb-kube-api-access-hhbts\") pod \"payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.142408 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.142298    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:37.179869 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.179812    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.313114 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.313076    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"]
Jul 29 22:03:37.314194 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:03:37.314138    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94dcabcb_054d_42ef_bbe0_5d37823ab154.slice/crio-d32a46bd7a8c8359509261a56643d06a54a6d4c0b5c18fbd5104e14d2a48e742 WatchSource:0}: Error finding container d32a46bd7a8c8359509261a56643d06a54a6d4c0b5c18fbd5104e14d2a48e742: Status 404 returned error can't find the container with id d32a46bd7a8c8359509261a56643d06a54a6d4c0b5c18fbd5104e14d2a48e742
Jul 29 22:03:37.351625 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.351538    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"]
Jul 29 22:03:37.353967 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:03:37.353944    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f8412b_bebe_4b58_85c7_e468c54466eb.slice/crio-120efc86706750616d03be268443f17248dcac3df2af023420a831073f6be906 WatchSource:0}: Error finding container 120efc86706750616d03be268443f17248dcac3df2af023420a831073f6be906: Status 404 returned error can't find the container with id 120efc86706750616d03be268443f17248dcac3df2af023420a831073f6be906
Jul 29 22:03:37.771924 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.771884    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl" event={"ID":"94dcabcb-054d-42ef-bbe0-5d37823ab154","Type":"ContainerStarted","Data":"52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f"}
Jul 29 22:03:37.772322 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.771958    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl" event={"ID":"94dcabcb-054d-42ef-bbe0-5d37823ab154","Type":"ContainerStarted","Data":"d32a46bd7a8c8359509261a56643d06a54a6d4c0b5c18fbd5104e14d2a48e742"}
Jul 29 22:03:37.772558 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.772442    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:37.774304 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.774275    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr" event={"ID":"85f8412b-bebe-4b58-85c7-e468c54466eb","Type":"ContainerStarted","Data":"78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199"}
Jul 29 22:03:37.774445 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.774326    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr" event={"ID":"85f8412b-bebe-4b58-85c7-e468c54466eb","Type":"ContainerStarted","Data":"120efc86706750616d03be268443f17248dcac3df2af023420a831073f6be906"}
Jul 29 22:03:37.774445 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.774359    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:03:37.812381 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.812326    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl" podStartSLOduration=1.8123121119999999 podStartE2EDuration="1.812312112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:03:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:03:37.810592124 +0000 UTC m=+2829.027369587" watchObservedRunningTime="2026-07-29 22:03:37.812312112 +0000 UTC m=+2829.029089574"
Jul 29 22:03:37.843199 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:37.843134    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr" podStartSLOduration=1.843119553 podStartE2EDuration="1.843119553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:03:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:03:37.840346078 +0000 UTC m=+2829.057123551" watchObservedRunningTime="2026-07-29 22:03:37.843119553 +0000 UTC m=+2829.059897016"
Jul 29 22:03:40.736359 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:40.736319    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:03:48.782154 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:48.782122    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:03:48.782854 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:03:48.782274    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:04:04.428790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:04.428755    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:04.947877 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:04.947837    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:05.399757 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:05.399676    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:05.510946 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:05.510909    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:08.511581 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:08.511538    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:08.637068 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:08.637030    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:09.708034 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:09.707995    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:10.080201 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.080104    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:10.499841 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.499805    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:10.605141 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.605100    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:10.681606 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.681571    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"]
Jul 29 22:04:10.684976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.684959    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.687517 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.687489    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-6ab119-dockercfg-gsjw9\""
Jul 29 22:04:10.688001 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.687982    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-6ab119\""
Jul 29 22:04:10.696030 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.696004    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"]
Jul 29 22:04:10.854421 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.854324    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9305bd07-2275-4198-934d-fb54c617ff3f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.854421 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.854374    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7wgj\" (UniqueName: \"kubernetes.io/projected/9305bd07-2275-4198-934d-fb54c617ff3f-kube-api-access-z7wgj\") pod \"payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.955619 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.955577    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9305bd07-2275-4198-934d-fb54c617ff3f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.955807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.955628    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7wgj\" (UniqueName: \"kubernetes.io/projected/9305bd07-2275-4198-934d-fb54c617ff3f-kube-api-access-z7wgj\") pod \"payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.956342 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.956321    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9305bd07-2275-4198-934d-fb54c617ff3f-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.965061 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.965024    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7wgj\" (UniqueName: \"kubernetes.io/projected/9305bd07-2275-4198-934d-fb54c617ff3f-kube-api-access-z7wgj\") pod \"payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") " pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:10.996620 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:10.996584    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:11.367396 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:04:11.367350    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9305bd07_2275_4198_934d_fb54c617ff3f.slice/crio-a2a10e01010b855658459852e0ae2f7cb141601027d8f065074668e6888ed3c2 WatchSource:0}: Error finding container a2a10e01010b855658459852e0ae2f7cb141601027d8f065074668e6888ed3c2: Status 404 returned error can't find the container with id a2a10e01010b855658459852e0ae2f7cb141601027d8f065074668e6888ed3c2
Jul 29 22:04:11.367728 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:11.367703    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"]
Jul 29 22:04:11.972211 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:11.972148    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh" event={"ID":"9305bd07-2275-4198-934d-fb54c617ff3f","Type":"ContainerStarted","Data":"8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63"}
Jul 29 22:04:11.972630 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:11.972241    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:11.972630 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:11.972263    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh" event={"ID":"9305bd07-2275-4198-934d-fb54c617ff3f","Type":"ContainerStarted","Data":"a2a10e01010b855658459852e0ae2f7cb141601027d8f065074668e6888ed3c2"}
Jul 29 22:04:12.033727 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:12.033665    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh" podStartSLOduration=2.033648613 podStartE2EDuration="2.033648613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:04:12.032525774 +0000 UTC m=+2863.249303236" watchObservedRunningTime="2026-07-29 22:04:12.033648613 +0000 UTC m=+2863.250426075"
Jul 29 22:04:15.144191 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:15.144149    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:04:22.980454 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:22.980367    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:04:33.566194 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:33.566140    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:52.466359 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.466315    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:52.497661 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.497623    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"]
Jul 29 22:04:52.497927 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.497898    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl" podUID="94dcabcb-054d-42ef-bbe0-5d37823ab154" containerName="payload-processing" containerID="cri-o://52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f" gracePeriod=30
Jul 29 22:04:52.508297 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.508266    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"]
Jul 29 22:04:52.508526 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.508500    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr" podUID="85f8412b-bebe-4b58-85c7-e468c54466eb" containerName="payload-pre-processing" containerID="cri-o://78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199" gracePeriod=30
Jul 29 22:04:52.551180 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.551126    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:52.631875 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.631835    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:52.703207 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.703145    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:52.795564 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.795533    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:04:52.805252 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.805218    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:04:52.930310 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.930275    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhbts\" (UniqueName: \"kubernetes.io/projected/85f8412b-bebe-4b58-85c7-e468c54466eb-kube-api-access-hhbts\") pod \"85f8412b-bebe-4b58-85c7-e468c54466eb\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") "
Jul 29 22:04:52.930528 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.930402    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f8412b-bebe-4b58-85c7-e468c54466eb-plugins-config-volume\") pod \"85f8412b-bebe-4b58-85c7-e468c54466eb\" (UID: \"85f8412b-bebe-4b58-85c7-e468c54466eb\") "
Jul 29 22:04:52.930528 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.930473    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp6ml\" (UniqueName: \"kubernetes.io/projected/94dcabcb-054d-42ef-bbe0-5d37823ab154-kube-api-access-rp6ml\") pod \"94dcabcb-054d-42ef-bbe0-5d37823ab154\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") "
Jul 29 22:04:52.930528 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.930518    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94dcabcb-054d-42ef-bbe0-5d37823ab154-plugins-config-volume\") pod \"94dcabcb-054d-42ef-bbe0-5d37823ab154\" (UID: \"94dcabcb-054d-42ef-bbe0-5d37823ab154\") "
Jul 29 22:04:52.930793 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.930770    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f8412b-bebe-4b58-85c7-e468c54466eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "85f8412b-bebe-4b58-85c7-e468c54466eb" (UID: "85f8412b-bebe-4b58-85c7-e468c54466eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:04:52.930988 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.930964    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94dcabcb-054d-42ef-bbe0-5d37823ab154-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "94dcabcb-054d-42ef-bbe0-5d37823ab154" (UID: "94dcabcb-054d-42ef-bbe0-5d37823ab154"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:04:52.932548 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.932516    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85f8412b-bebe-4b58-85c7-e468c54466eb-kube-api-access-hhbts" (OuterVolumeSpecName: "kube-api-access-hhbts") pod "85f8412b-bebe-4b58-85c7-e468c54466eb" (UID: "85f8412b-bebe-4b58-85c7-e468c54466eb"). InnerVolumeSpecName "kube-api-access-hhbts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:04:52.932746 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:52.932721    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94dcabcb-054d-42ef-bbe0-5d37823ab154-kube-api-access-rp6ml" (OuterVolumeSpecName: "kube-api-access-rp6ml") pod "94dcabcb-054d-42ef-bbe0-5d37823ab154" (UID: "94dcabcb-054d-42ef-bbe0-5d37823ab154"). InnerVolumeSpecName "kube-api-access-rp6ml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:04:53.031448 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.031353    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85f8412b-bebe-4b58-85c7-e468c54466eb-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:04:53.031448 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.031392    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp6ml\" (UniqueName: \"kubernetes.io/projected/94dcabcb-054d-42ef-bbe0-5d37823ab154-kube-api-access-rp6ml\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:04:53.031448 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.031408    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/94dcabcb-054d-42ef-bbe0-5d37823ab154-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:04:53.031448 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.031416    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhbts\" (UniqueName: \"kubernetes.io/projected/85f8412b-bebe-4b58-85c7-e468c54466eb-kube-api-access-hhbts\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:04:53.095545 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.095502    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:53.206127 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.206095    2572 generic.go:356] "Generic (PLEG): container finished" podID="94dcabcb-054d-42ef-bbe0-5d37823ab154" containerID="52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f" exitCode=0
Jul 29 22:04:53.206331 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.206157    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"
Jul 29 22:04:53.206331 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.206192    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl" event={"ID":"94dcabcb-054d-42ef-bbe0-5d37823ab154","Type":"ContainerDied","Data":"52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f"}
Jul 29 22:04:53.206331 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.206282    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl" event={"ID":"94dcabcb-054d-42ef-bbe0-5d37823ab154","Type":"ContainerDied","Data":"d32a46bd7a8c8359509261a56643d06a54a6d4c0b5c18fbd5104e14d2a48e742"}
Jul 29 22:04:53.206331 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.206322    2572 scope.go:117] "RemoveContainer" containerID="52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f"
Jul 29 22:04:53.207621 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.207595    2572 generic.go:356] "Generic (PLEG): container finished" podID="85f8412b-bebe-4b58-85c7-e468c54466eb" containerID="78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199" exitCode=0
Jul 29 22:04:53.207702 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.207656    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"
Jul 29 22:04:53.207766 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.207692    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr" event={"ID":"85f8412b-bebe-4b58-85c7-e468c54466eb","Type":"ContainerDied","Data":"78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199"}
Jul 29 22:04:53.207766 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.207756    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr" event={"ID":"85f8412b-bebe-4b58-85c7-e468c54466eb","Type":"ContainerDied","Data":"120efc86706750616d03be268443f17248dcac3df2af023420a831073f6be906"}
Jul 29 22:04:53.216775 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.216759    2572 scope.go:117] "RemoveContainer" containerID="52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f"
Jul 29 22:04:53.217085 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:04:53.217059    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f\": container with ID starting with 52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f not found: ID does not exist" containerID="52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f"
Jul 29 22:04:53.217206 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.217097    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f"} err="failed to get container status \"52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f\": rpc error: code = NotFound desc = could not find container \"52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f\": container with ID starting with 52707673e069193f690aad84716611087fc8637f9e0225248ad5384ee7d9ae8f not found: ID does not exist"
Jul 29 22:04:53.217206 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.217122    2572 scope.go:117] "RemoveContainer" containerID="78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199"
Jul 29 22:04:53.225510 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.225495    2572 scope.go:117] "RemoveContainer" containerID="78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199"
Jul 29 22:04:53.225768 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:04:53.225749    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199\": container with ID starting with 78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199 not found: ID does not exist" containerID="78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199"
Jul 29 22:04:53.225818 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.225774    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199"} err="failed to get container status \"78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199\": rpc error: code = NotFound desc = could not find container \"78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199\": container with ID starting with 78ad912c96915eb820d2ce933b42e735e85d3737aac3ee84b700416f8acef199 not found: ID does not exist"
Jul 29 22:04:53.232849 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.232829    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"]
Jul 29 22:04:53.240849 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.240830    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c8e5f0-75f5b84964-cckvr"]
Jul 29 22:04:53.256086 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.256064    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"]
Jul 29 22:04:53.258238 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.258221    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-c8e5f0-8fcf5c454-7cjtl"]
Jul 29 22:04:53.305362 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.305283    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f8412b-bebe-4b58-85c7-e468c54466eb" path="/var/lib/kubelet/pods/85f8412b-bebe-4b58-85c7-e468c54466eb/volumes"
Jul 29 22:04:53.305689 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:53.305673    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94dcabcb-054d-42ef-bbe0-5d37823ab154" path="/var/lib/kubelet/pods/94dcabcb-054d-42ef-bbe0-5d37823ab154/volumes"
Jul 29 22:04:54.742425 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:54.742383    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:04:59.071206 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.071153    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:59.167486 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.167449    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:59.413690 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.413604    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:59.642421 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.642388    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:04:59.767424 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767387    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"]
Jul 29 22:04:59.767821 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767806    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94dcabcb-054d-42ef-bbe0-5d37823ab154" containerName="payload-processing"
Jul 29 22:04:59.767874 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767822    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="94dcabcb-054d-42ef-bbe0-5d37823ab154" containerName="payload-processing"
Jul 29 22:04:59.767874 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767841    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f8412b-bebe-4b58-85c7-e468c54466eb" containerName="payload-pre-processing"
Jul 29 22:04:59.767874 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767848    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f8412b-bebe-4b58-85c7-e468c54466eb" containerName="payload-pre-processing"
Jul 29 22:04:59.767981 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767917    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f8412b-bebe-4b58-85c7-e468c54466eb" containerName="payload-pre-processing"
Jul 29 22:04:59.767981 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.767927    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="94dcabcb-054d-42ef-bbe0-5d37823ab154" containerName="payload-processing"
Jul 29 22:04:59.778219 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.778155    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:04:59.780762 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.780735    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:04:59.780977 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.780898    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxbps\""
Jul 29 22:04:59.780977 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.780968    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"]
Jul 29 22:04:59.898089 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.898045    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e129942-b2e5-485c-a4e7-838be6a42850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6ab119-98hlp\" (UID: \"2e129942-b2e5-485c-a4e7-838be6a42850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:04:59.999466 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:04:59.999413    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e129942-b2e5-485c-a4e7-838be6a42850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6ab119-98hlp\" (UID: \"2e129942-b2e5-485c-a4e7-838be6a42850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:05:00.000082 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:00.000059    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e129942-b2e5-485c-a4e7-838be6a42850-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-6ab119-98hlp\" (UID: \"2e129942-b2e5-485c-a4e7-838be6a42850\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:05:00.090669 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:00.090573    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:05:00.236532 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:00.236503    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"]
Jul 29 22:05:00.239429 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:05:00.239386    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e129942_b2e5_485c_a4e7_838be6a42850.slice/crio-03e527ef3963ab41575fd8b9053ac1d3020464cff323bac5357eacd0f1a77989 WatchSource:0}: Error finding container 03e527ef3963ab41575fd8b9053ac1d3020464cff323bac5357eacd0f1a77989: Status 404 returned error can't find the container with id 03e527ef3963ab41575fd8b9053ac1d3020464cff323bac5357eacd0f1a77989
Jul 29 22:05:00.244486 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:00.244463    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 22:05:00.255658 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:00.255628    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp" event={"ID":"2e129942-b2e5-485c-a4e7-838be6a42850","Type":"ContainerStarted","Data":"03e527ef3963ab41575fd8b9053ac1d3020464cff323bac5357eacd0f1a77989"}
Jul 29 22:05:04.283804 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:04.283766    2572 generic.go:356] "Generic (PLEG): container finished" podID="2e129942-b2e5-485c-a4e7-838be6a42850" containerID="5e3738ee931be67d54e0db78360838cd6e78dd3972050f18b4693ab92765ec6e" exitCode=0
Jul 29 22:05:04.284247 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:04.283828    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp" event={"ID":"2e129942-b2e5-485c-a4e7-838be6a42850","Type":"ContainerDied","Data":"5e3738ee931be67d54e0db78360838cd6e78dd3972050f18b4693ab92765ec6e"}
Jul 29 22:05:05.430949 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:05.430921    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:05:05.441149 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:05.441126    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e129942-b2e5-485c-a4e7-838be6a42850-maas-api-service-ca\") pod \"2e129942-b2e5-485c-a4e7-838be6a42850\" (UID: \"2e129942-b2e5-485c-a4e7-838be6a42850\") "
Jul 29 22:05:05.441506 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:05.441487    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e129942-b2e5-485c-a4e7-838be6a42850-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e129942-b2e5-485c-a4e7-838be6a42850" (UID: "2e129942-b2e5-485c-a4e7-838be6a42850"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:05:05.542527 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:05.542450    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e129942-b2e5-485c-a4e7-838be6a42850-maas-api-service-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:05:06.295835 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:06.295804    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"
Jul 29 22:05:06.296020 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:06.295840    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp" event={"ID":"2e129942-b2e5-485c-a4e7-838be6a42850","Type":"ContainerDied","Data":"03e527ef3963ab41575fd8b9053ac1d3020464cff323bac5357eacd0f1a77989"}
Jul 29 22:05:06.296020 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:06.295911    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03e527ef3963ab41575fd8b9053ac1d3020464cff323bac5357eacd0f1a77989"
Jul 29 22:05:15.544085 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.544051    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"]
Jul 29 22:05:15.544556 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.544330    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh" podUID="9305bd07-2275-4198-934d-fb54c617ff3f" containerName="payload-processing" containerID="cri-o://8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63" gracePeriod=30
Jul 29 22:05:15.557109 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.557060    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:15.648789 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.648743    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:15.781569 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.781531    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"]
Jul 29 22:05:15.783994 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.783966    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-6ab119-98hlp"]
Jul 29 22:05:15.803656 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.803593    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:05:15.834193 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.834144    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:15.842472 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.842448    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7wgj\" (UniqueName: \"kubernetes.io/projected/9305bd07-2275-4198-934d-fb54c617ff3f-kube-api-access-z7wgj\") pod \"9305bd07-2275-4198-934d-fb54c617ff3f\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") "
Jul 29 22:05:15.842622 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.842504    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9305bd07-2275-4198-934d-fb54c617ff3f-plugins-config-volume\") pod \"9305bd07-2275-4198-934d-fb54c617ff3f\" (UID: \"9305bd07-2275-4198-934d-fb54c617ff3f\") "
Jul 29 22:05:15.842888 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.842863    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9305bd07-2275-4198-934d-fb54c617ff3f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9305bd07-2275-4198-934d-fb54c617ff3f" (UID: "9305bd07-2275-4198-934d-fb54c617ff3f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:05:15.844783 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.844751    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9305bd07-2275-4198-934d-fb54c617ff3f-kube-api-access-z7wgj" (OuterVolumeSpecName: "kube-api-access-z7wgj") pod "9305bd07-2275-4198-934d-fb54c617ff3f" (UID: "9305bd07-2275-4198-934d-fb54c617ff3f"). InnerVolumeSpecName "kube-api-access-z7wgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:05:15.943418 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.943379    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7wgj\" (UniqueName: \"kubernetes.io/projected/9305bd07-2275-4198-934d-fb54c617ff3f-kube-api-access-z7wgj\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:05:15.943418 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:15.943414    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9305bd07-2275-4198-934d-fb54c617ff3f-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:05:16.235577 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.235535    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:16.351783 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.351744    2572 generic.go:356] "Generic (PLEG): container finished" podID="9305bd07-2275-4198-934d-fb54c617ff3f" containerID="8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63" exitCode=0
Jul 29 22:05:16.351998 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.351788    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"
Jul 29 22:05:16.351998 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.351834    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh" event={"ID":"9305bd07-2275-4198-934d-fb54c617ff3f","Type":"ContainerDied","Data":"8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63"}
Jul 29 22:05:16.351998 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.351893    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh" event={"ID":"9305bd07-2275-4198-934d-fb54c617ff3f","Type":"ContainerDied","Data":"a2a10e01010b855658459852e0ae2f7cb141601027d8f065074668e6888ed3c2"}
Jul 29 22:05:16.351998 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.351923    2572 scope.go:117] "RemoveContainer" containerID="8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63"
Jul 29 22:05:16.363352 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.363330    2572 scope.go:117] "RemoveContainer" containerID="8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63"
Jul 29 22:05:16.363713 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:05:16.363692    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63\": container with ID starting with 8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63 not found: ID does not exist" containerID="8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63"
Jul 29 22:05:16.363774 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.363726    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63"} err="failed to get container status \"8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63\": rpc error: code = NotFound desc = could not find container \"8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63\": container with ID starting with 8339a29f722146385dc899ee97973a85b8c360b42f3ed9d4238637da9359ba63 not found: ID does not exist"
Jul 29 22:05:16.378282 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.378239    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"]
Jul 29 22:05:16.383074 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:16.383039    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6ab119-6466799dc4-p2flh"]
Jul 29 22:05:17.312480 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:17.312437    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e129942-b2e5-485c-a4e7-838be6a42850" path="/var/lib/kubelet/pods/2e129942-b2e5-485c-a4e7-838be6a42850/volumes"
Jul 29 22:05:17.312946 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:17.312928    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9305bd07-2275-4198-934d-fb54c617ff3f" path="/var/lib/kubelet/pods/9305bd07-2275-4198-934d-fb54c617ff3f/volumes"
Jul 29 22:05:20.134417 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:20.134383    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:05:22.719740 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:22.719708    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:22.807972 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:22.807930    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:23.021205 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:23.021106    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:23.277986 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:23.277891    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:23.719686 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:23.719643    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:24.050545 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:24.050454    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:24.192512 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:24.192473    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:26.062622 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:26.062590    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:26.995499 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:26.995432    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:27.081079 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:27.081043    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:28.143236 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:28.143196    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:28.784457 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:28.784410    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:28.903681 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:28.903645    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:29.029790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.029759    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"]
Jul 29 22:05:29.030154 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.030142    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e129942-b2e5-485c-a4e7-838be6a42850" containerName="revoke-keys"
Jul 29 22:05:29.030216 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.030156    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e129942-b2e5-485c-a4e7-838be6a42850" containerName="revoke-keys"
Jul 29 22:05:29.030216 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.030192    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9305bd07-2275-4198-934d-fb54c617ff3f" containerName="payload-processing"
Jul 29 22:05:29.030216 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.030199    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9305bd07-2275-4198-934d-fb54c617ff3f" containerName="payload-processing"
Jul 29 22:05:29.030313 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.030275    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e129942-b2e5-485c-a4e7-838be6a42850" containerName="revoke-keys"
Jul 29 22:05:29.030313 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.030282    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="9305bd07-2275-4198-934d-fb54c617ff3f" containerName="payload-processing"
Jul 29 22:05:29.034750 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.034695    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.038401 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.038379    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-c9eccd-dockercfg-rps8d\""
Jul 29 22:05:29.038745 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.038722    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-c9eccd\""
Jul 29 22:05:29.051241 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.051212    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"]
Jul 29 22:05:29.168682 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.168645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d724c10-f9d0-49ff-b634-c2b4898016ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.169088 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.168722    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzfxs\" (UniqueName: \"kubernetes.io/projected/8d724c10-f9d0-49ff-b634-c2b4898016ce-kube-api-access-mzfxs\") pod \"payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.269753 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.269717    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d724c10-f9d0-49ff-b634-c2b4898016ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.269940 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.269815    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzfxs\" (UniqueName: \"kubernetes.io/projected/8d724c10-f9d0-49ff-b634-c2b4898016ce-kube-api-access-mzfxs\") pod \"payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.273828 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.273800    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-c9eccd\""
Jul 29 22:05:29.280207 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.280189    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzfxs\" (UniqueName: \"kubernetes.io/projected/8d724c10-f9d0-49ff-b634-c2b4898016ce-kube-api-access-mzfxs\") pod \"payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.280706 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.280690    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d724c10-f9d0-49ff-b634-c2b4898016ce-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") " pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.347873 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.347815    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-c9eccd-dockercfg-rps8d\""
Jul 29 22:05:29.355072 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.355056    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:29.502997 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:29.502970    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"]
Jul 29 22:05:29.504592 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:05:29.504556    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d724c10_f9d0_49ff_b634_c2b4898016ce.slice/crio-ece9766478dd0b1fc1eb082671ff97f10ba3b59a9c4fb2783ed111579e996fb7 WatchSource:0}: Error finding container ece9766478dd0b1fc1eb082671ff97f10ba3b59a9c4fb2783ed111579e996fb7: Status 404 returned error can't find the container with id ece9766478dd0b1fc1eb082671ff97f10ba3b59a9c4fb2783ed111579e996fb7
Jul 29 22:05:30.443659 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:30.443623    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp" event={"ID":"8d724c10-f9d0-49ff-b634-c2b4898016ce","Type":"ContainerStarted","Data":"bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e"}
Jul 29 22:05:30.444032 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:30.443678    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp" event={"ID":"8d724c10-f9d0-49ff-b634-c2b4898016ce","Type":"ContainerStarted","Data":"ece9766478dd0b1fc1eb082671ff97f10ba3b59a9c4fb2783ed111579e996fb7"}
Jul 29 22:05:30.444032 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:30.443709    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:30.462301 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:30.462249    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp" podStartSLOduration=1.462234011 podStartE2EDuration="1.462234011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:05:30.460412619 +0000 UTC m=+2941.677190080" watchObservedRunningTime="2026-07-29 22:05:30.462234011 +0000 UTC m=+2941.679011474"
Jul 29 22:05:31.338060 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:31.338023    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:05:41.451910 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:41.451881    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:05:51.986529 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:51.986441    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:52.354116 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:52.354027    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:52.751588 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:52.751550    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:52.919807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:52.919763    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:56.123248 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:56.123125    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:56.260063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:56.260026    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:57.373985 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:57.373929    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:57.855494 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:57.855457    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:58.030795 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.030745    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:58.123237 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.123142    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:05:58.182632 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.182593    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"]
Jul 29 22:05:58.185201 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.185163    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.188566 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.188545    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-ca895e\""
Jul 29 22:05:58.189585 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.189568    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-ca895e-dockercfg-xsfn2\""
Jul 29 22:05:58.204127 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.204109    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"]
Jul 29 22:05:58.245334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.245305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d7826d-1a4d-4bfc-a71f-899088479944-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.245495 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.245367    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhdnd\" (UniqueName: \"kubernetes.io/projected/82d7826d-1a4d-4bfc-a71f-899088479944-kube-api-access-zhdnd\") pod \"payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.346435 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.346399    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhdnd\" (UniqueName: \"kubernetes.io/projected/82d7826d-1a4d-4bfc-a71f-899088479944-kube-api-access-zhdnd\") pod \"payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.346623 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.346486    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d7826d-1a4d-4bfc-a71f-899088479944-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.347196 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.347147    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d7826d-1a4d-4bfc-a71f-899088479944-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.359790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.359763    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhdnd\" (UniqueName: \"kubernetes.io/projected/82d7826d-1a4d-4bfc-a71f-899088479944-kube-api-access-zhdnd\") pod \"payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.495452 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.495413    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:58.847043 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:58.847015    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"]
Jul 29 22:05:58.849129 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:05:58.849101    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d7826d_1a4d_4bfc_a71f_899088479944.slice/crio-ed3a305db7717fe2319c676714ede2e04f4de188c3302f4a59e7e924e458b18b WatchSource:0}: Error finding container ed3a305db7717fe2319c676714ede2e04f4de188c3302f4a59e7e924e458b18b: Status 404 returned error can't find the container with id ed3a305db7717fe2319c676714ede2e04f4de188c3302f4a59e7e924e458b18b
Jul 29 22:05:59.611050 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:59.611004    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps" event={"ID":"82d7826d-1a4d-4bfc-a71f-899088479944","Type":"ContainerStarted","Data":"6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88"}
Jul 29 22:05:59.611490 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:59.611065    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps" event={"ID":"82d7826d-1a4d-4bfc-a71f-899088479944","Type":"ContainerStarted","Data":"ed3a305db7717fe2319c676714ede2e04f4de188c3302f4a59e7e924e458b18b"}
Jul 29 22:05:59.611490 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:59.611099    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:05:59.636884 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:05:59.636829    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps" podStartSLOduration=1.636809808 podStartE2EDuration="1.636809808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:05:59.635995876 +0000 UTC m=+2970.852773338" watchObservedRunningTime="2026-07-29 22:05:59.636809808 +0000 UTC m=+2970.853587270"
Jul 29 22:06:03.140161 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:03.140124    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:06:10.619210 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:10.619160    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:06:21.150815 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:21.150771    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:06:32.298127 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.298093    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"]
Jul 29 22:06:32.300956 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.300934    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.304307 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.304289    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"kube-root-ca.crt\""
Jul 29 22:06:32.304575 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.304549    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"openshift-service-ca.crt\""
Jul 29 22:06:32.304575 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.304550    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"default-dockercfg-gkfjv\""
Jul 29 22:06:32.305125 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.305111    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"auth-test-model-c9eccd-kserve-self-signed-certs\""
Jul 29 22:06:32.313484 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.313457    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"]
Jul 29 22:06:32.343594 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.343562    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:06:32.371370 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.371341    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-tmp-dir\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.371507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.371382    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdb909-6a79-481e-9dfa-44074b350e93-tls-certs\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.371507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.371420    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-home\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.371507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.371452    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-model-cache\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.371690 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.371587    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swjg8\" (UniqueName: \"kubernetes.io/projected/19fdb909-6a79-481e-9dfa-44074b350e93-kube-api-access-swjg8\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.371690 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.371646    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-dshm\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472379 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472334    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-home\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472379 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472377    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-model-cache\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472658 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472425    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swjg8\" (UniqueName: \"kubernetes.io/projected/19fdb909-6a79-481e-9dfa-44074b350e93-kube-api-access-swjg8\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472658 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472456    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-dshm\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472658 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472500    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-tmp-dir\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472658 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472518    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdb909-6a79-481e-9dfa-44074b350e93-tls-certs\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472879 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472840    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-home\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472879 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472866    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-model-cache\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.472959 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.472925    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-tmp-dir\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.474826 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.474803    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-dshm\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.475264 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.475241    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdb909-6a79-481e-9dfa-44074b350e93-tls-certs\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.486540 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.486503    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swjg8\" (UniqueName: \"kubernetes.io/projected/19fdb909-6a79-481e-9dfa-44074b350e93-kube-api-access-swjg8\") pod \"auth-test-model-c9eccd-kserve-6b4498667-5hc24\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") " pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.610778 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.610679    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:32.757052 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.757029    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"]
Jul 29 22:06:32.758651 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:06:32.758628    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19fdb909_6a79_481e_9dfa_44074b350e93.slice/crio-c71a4753c1095632ef5d0f5e673d9281a75ac009607ba39e1d3a44db687e8411 WatchSource:0}: Error finding container c71a4753c1095632ef5d0f5e673d9281a75ac009607ba39e1d3a44db687e8411: Status 404 returned error can't find the container with id c71a4753c1095632ef5d0f5e673d9281a75ac009607ba39e1d3a44db687e8411
Jul 29 22:06:32.802891 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:32.802862    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24" event={"ID":"19fdb909-6a79-481e-9dfa-44074b350e93","Type":"ContainerStarted","Data":"c71a4753c1095632ef5d0f5e673d9281a75ac009607ba39e1d3a44db687e8411"}
Jul 29 22:06:33.810247 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:33.810209    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24" event={"ID":"19fdb909-6a79-481e-9dfa-44074b350e93","Type":"ContainerStarted","Data":"856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943"}
Jul 29 22:06:33.831063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:33.831006    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24" podStartSLOduration=1.830990559 podStartE2EDuration="1.830990559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:06:33.829828139 +0000 UTC m=+3005.046605602" watchObservedRunningTime="2026-07-29 22:06:33.830990559 +0000 UTC m=+3005.047768020"
Jul 29 22:06:35.833336 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:35.833294    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:06:42.611897 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:42.611840    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:42.611897 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:42.611906    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:42.624600 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:42.624567    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:06:42.870225 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:06:42.870123    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:07:22.532722 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.532634    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"]
Jul 29 22:07:22.535691 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.535668    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.539279 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.539249    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"openshift-service-ca.crt\""
Jul 29 22:07:22.539279 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.539276    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"kube-root-ca.crt\""
Jul 29 22:07:22.539494 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.539284    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"auth-test-model-ca895e-kserve-self-signed-certs\""
Jul 29 22:07:22.539494 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.539285    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"default-dockercfg-hbjlz\""
Jul 29 22:07:22.552074 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.552040    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"]
Jul 29 22:07:22.585778 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.585217    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:07:22.638967 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.638929    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-dshm\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.639149 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.638997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fed3a72-695a-4f84-be34-453da65962fa-tls-certs\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.639149 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.639044    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-tmp-dir\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.639265 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.639150    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-home\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.639265 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.639228    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-model-cache\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.639334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.639272    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6t7p\" (UniqueName: \"kubernetes.io/projected/4fed3a72-695a-4f84-be34-453da65962fa-kube-api-access-x6t7p\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740060 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740011    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-home\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740060 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740057    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-model-cache\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740382 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740103    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6t7p\" (UniqueName: \"kubernetes.io/projected/4fed3a72-695a-4f84-be34-453da65962fa-kube-api-access-x6t7p\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740382 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740142    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-dshm\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740382 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740230    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fed3a72-695a-4f84-be34-453da65962fa-tls-certs\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740382 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740291    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-tmp-dir\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740604 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740486    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-home\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740604 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740501    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-model-cache\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.740718 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.740682    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-tmp-dir\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.742433 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.742411    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-dshm\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.742759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.742739    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fed3a72-695a-4f84-be34-453da65962fa-tls-certs\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.749412 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.749390    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6t7p\" (UniqueName: \"kubernetes.io/projected/4fed3a72-695a-4f84-be34-453da65962fa-kube-api-access-x6t7p\") pod \"auth-test-model-ca895e-kserve-67f99b8c7f-f47hv\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") " pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.845883 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.845794    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:22.990707 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:22.990674    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"]
Jul 29 22:07:22.993970 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:07:22.993938    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fed3a72_695a_4f84_be34_453da65962fa.slice/crio-fb83c2f89b33874da6abf142904f6c3d3b0025ea3b84a9384efd4937eb2ff912 WatchSource:0}: Error finding container fb83c2f89b33874da6abf142904f6c3d3b0025ea3b84a9384efd4937eb2ff912: Status 404 returned error can't find the container with id fb83c2f89b33874da6abf142904f6c3d3b0025ea3b84a9384efd4937eb2ff912
Jul 29 22:07:23.085388 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:23.085341    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv" event={"ID":"4fed3a72-695a-4f84-be34-453da65962fa","Type":"ContainerStarted","Data":"8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2"}
Jul 29 22:07:23.085597 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:23.085402    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv" event={"ID":"4fed3a72-695a-4f84-be34-453da65962fa","Type":"ContainerStarted","Data":"fb83c2f89b33874da6abf142904f6c3d3b0025ea3b84a9384efd4937eb2ff912"}
Jul 29 22:07:23.111113 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:23.111009    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv" podStartSLOduration=1.110993145 podStartE2EDuration="1.110993145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:07:23.109828315 +0000 UTC m=+3054.326605780" watchObservedRunningTime="2026-07-29 22:07:23.110993145 +0000 UTC m=+3054.327770606"
Jul 29 22:07:26.238031 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:26.237988    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:07:32.846560 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:32.846515    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:32.846560 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:32.846576    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:32.859111 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:32.859087    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:33.153017 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:33.152937    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:07:58.246207 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:07:58.246151    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:08:25.125607 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:08:25.125570    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:08:43.645376 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:08:43.645338    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:09:05.594102 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.594013    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qznhv"]
Jul 29 22:09:05.597310 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.597284    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:05.618218 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.618184    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qznhv"]
Jul 29 22:09:05.719331 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.719279    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrsm5\" (UniqueName: \"kubernetes.io/projected/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5-kube-api-access-lrsm5\") pod \"kuadrant-operator-catalog-qznhv\" (UID: \"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5\") " pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:05.820470 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.820423    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrsm5\" (UniqueName: \"kubernetes.io/projected/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5-kube-api-access-lrsm5\") pod \"kuadrant-operator-catalog-qznhv\" (UID: \"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5\") " pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:05.829879 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.829845    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrsm5\" (UniqueName: \"kubernetes.io/projected/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5-kube-api-access-lrsm5\") pod \"kuadrant-operator-catalog-qznhv\" (UID: \"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5\") " pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:05.907616 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:05.907533    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:06.042678 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:06.042655    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qznhv"]
Jul 29 22:09:06.044798 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:09:06.044768    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea8c2ef_d129_45ee_ad48_a0e6b86e03b5.slice/crio-041b7826cfc91b4991717157ecdc76d3cd46d926c3a12cd777beeb98164fee92 WatchSource:0}: Error finding container 041b7826cfc91b4991717157ecdc76d3cd46d926c3a12cd777beeb98164fee92: Status 404 returned error can't find the container with id 041b7826cfc91b4991717157ecdc76d3cd46d926c3a12cd777beeb98164fee92
Jul 29 22:09:06.671582 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:06.671537    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qznhv" event={"ID":"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5","Type":"ContainerStarted","Data":"d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad"}
Jul 29 22:09:06.671992 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:06.671636    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qznhv" event={"ID":"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5","Type":"ContainerStarted","Data":"041b7826cfc91b4991717157ecdc76d3cd46d926c3a12cd777beeb98164fee92"}
Jul 29 22:09:06.691219 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:06.691145    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qznhv" podStartSLOduration=1.251561416 podStartE2EDuration="1.691128653s" totalImagesPullingTime="439.567237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:09:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:09:06.688242132 +0000 UTC m=+3157.905019598" watchObservedRunningTime="2026-07-29 22:09:06.691128653 +0000 UTC m=+3157.907906144"
Jul 29 22:09:11.231275 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:11.231238    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:09:15.908184 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:15.908127    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:15.908677 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:15.908214    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:15.931656 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:15.931626    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:16.750733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:16.750706    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:16.795251 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:16.795212    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qznhv"]
Jul 29 22:09:18.740405 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:18.740357    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qznhv" podUID="7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" containerName="registry-server" containerID="cri-o://d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad" gracePeriod=2
Jul 29 22:09:19.000914 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.000855    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:19.154853 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.154817    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrsm5\" (UniqueName: \"kubernetes.io/projected/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5-kube-api-access-lrsm5\") pod \"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5\" (UID: \"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5\") "
Jul 29 22:09:19.156928 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.156895    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5-kube-api-access-lrsm5" (OuterVolumeSpecName: "kube-api-access-lrsm5") pod "7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" (UID: "7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5"). InnerVolumeSpecName "kube-api-access-lrsm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:09:19.256604 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.256499    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrsm5\" (UniqueName: \"kubernetes.io/projected/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5-kube-api-access-lrsm5\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:09:19.746698 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.746662    2572 generic.go:356] "Generic (PLEG): container finished" podID="7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" containerID="d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad" exitCode=0
Jul 29 22:09:19.747130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.746697    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qznhv" event={"ID":"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5","Type":"ContainerDied","Data":"d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad"}
Jul 29 22:09:19.747130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.746708    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qznhv"
Jul 29 22:09:19.747130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.746760    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qznhv" event={"ID":"7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5","Type":"ContainerDied","Data":"041b7826cfc91b4991717157ecdc76d3cd46d926c3a12cd777beeb98164fee92"}
Jul 29 22:09:19.747130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.746784    2572 scope.go:117] "RemoveContainer" containerID="d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad"
Jul 29 22:09:19.756579 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.756555    2572 scope.go:117] "RemoveContainer" containerID="d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad"
Jul 29 22:09:19.756839 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:09:19.756818    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad\": container with ID starting with d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad not found: ID does not exist" containerID="d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad"
Jul 29 22:09:19.756886 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.756848    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad"} err="failed to get container status \"d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad\": rpc error: code = NotFound desc = could not find container \"d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad\": container with ID starting with d8d7c3a5b2ed0561011bdc1721ec792b14c9bb847b1633835bdedae7486c91ad not found: ID does not exist"
Jul 29 22:09:19.767844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.767821    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qznhv"]
Jul 29 22:09:19.772019 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:19.771996    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qznhv"]
Jul 29 22:09:21.304713 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:21.304681    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" path="/var/lib/kubelet/pods/7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5/volumes"
Jul 29 22:09:29.439852 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:29.439817    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:09:54.832316 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:54.832277    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:09:57.650027 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:57.649989    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:09:59.414486 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:09:59.414438    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:10:11.740490 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:10:11.740448    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:10:39.371994 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:10:39.371889    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:10:57.425825 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:10:57.425787    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:11:22.929302 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:11:22.929264    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:11:27.369695 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:11:27.369660    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:11:27.458602 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:11:27.458567    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:11:29.760960 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:11:29.760926    2572 scope.go:117] "RemoveContainer" containerID="5e3738ee931be67d54e0db78360838cd6e78dd3972050f18b4693ab92765ec6e"
Jul 29 22:11:40.931761 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:11:40.931721    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:12:06.437065 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:06.436987    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:12:10.065851 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:10.065808    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"]
Jul 29 22:12:10.172880 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:10.172846    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:10.936758 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:10.936723    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"]
Jul 29 22:12:10.968748 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:10.968714    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:22.729662 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:22.729618    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:12:25.085354 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.085308    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24" podUID="19fdb909-6a79-481e-9dfa-44074b350e93" containerName="main" containerID="cri-o://856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943" gracePeriod=45
Jul 29 22:12:25.344614 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.344551    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:12:25.445008 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.444976    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-home\") pod \"19fdb909-6a79-481e-9dfa-44074b350e93\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") "
Jul 29 22:12:25.445197 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445023    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdb909-6a79-481e-9dfa-44074b350e93-tls-certs\") pod \"19fdb909-6a79-481e-9dfa-44074b350e93\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") "
Jul 29 22:12:25.445197 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445069    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swjg8\" (UniqueName: \"kubernetes.io/projected/19fdb909-6a79-481e-9dfa-44074b350e93-kube-api-access-swjg8\") pod \"19fdb909-6a79-481e-9dfa-44074b350e93\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") "
Jul 29 22:12:25.445197 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445117    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-model-cache\") pod \"19fdb909-6a79-481e-9dfa-44074b350e93\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") "
Jul 29 22:12:25.445197 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445148    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-dshm\") pod \"19fdb909-6a79-481e-9dfa-44074b350e93\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") "
Jul 29 22:12:25.445431 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445244    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-tmp-dir\") pod \"19fdb909-6a79-481e-9dfa-44074b350e93\" (UID: \"19fdb909-6a79-481e-9dfa-44074b350e93\") "
Jul 29 22:12:25.445593 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445233    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-home" (OuterVolumeSpecName: "home") pod "19fdb909-6a79-481e-9dfa-44074b350e93" (UID: "19fdb909-6a79-481e-9dfa-44074b350e93"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:25.445733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445572    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-model-cache" (OuterVolumeSpecName: "model-cache") pod "19fdb909-6a79-481e-9dfa-44074b350e93" (UID: "19fdb909-6a79-481e-9dfa-44074b350e93"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:25.445733 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.445587    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19fdb909-6a79-481e-9dfa-44074b350e93" (UID: "19fdb909-6a79-481e-9dfa-44074b350e93"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:25.447354 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.447331    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-dshm" (OuterVolumeSpecName: "dshm") pod "19fdb909-6a79-481e-9dfa-44074b350e93" (UID: "19fdb909-6a79-481e-9dfa-44074b350e93"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:25.447464 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.447382    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19fdb909-6a79-481e-9dfa-44074b350e93-kube-api-access-swjg8" (OuterVolumeSpecName: "kube-api-access-swjg8") pod "19fdb909-6a79-481e-9dfa-44074b350e93" (UID: "19fdb909-6a79-481e-9dfa-44074b350e93"). InnerVolumeSpecName "kube-api-access-swjg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:12:25.447464 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.447423    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19fdb909-6a79-481e-9dfa-44074b350e93-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19fdb909-6a79-481e-9dfa-44074b350e93" (UID: "19fdb909-6a79-481e-9dfa-44074b350e93"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:12:25.546407 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.546365    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:25.546407 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.546402    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fdb909-6a79-481e-9dfa-44074b350e93-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:25.546407 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.546414    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swjg8\" (UniqueName: \"kubernetes.io/projected/19fdb909-6a79-481e-9dfa-44074b350e93-kube-api-access-swjg8\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:25.546727 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.546424    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:25.546727 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.546433    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:25.546727 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.546440    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19fdb909-6a79-481e-9dfa-44074b350e93-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:25.827941 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.827900    2572 generic.go:356] "Generic (PLEG): container finished" podID="19fdb909-6a79-481e-9dfa-44074b350e93" containerID="856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943" exitCode=0
Jul 29 22:12:25.828126 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.827949    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"
Jul 29 22:12:25.828126 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.827982    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24" event={"ID":"19fdb909-6a79-481e-9dfa-44074b350e93","Type":"ContainerDied","Data":"856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943"}
Jul 29 22:12:25.828126 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.828030    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24" event={"ID":"19fdb909-6a79-481e-9dfa-44074b350e93","Type":"ContainerDied","Data":"c71a4753c1095632ef5d0f5e673d9281a75ac009607ba39e1d3a44db687e8411"}
Jul 29 22:12:25.828126 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.828048    2572 scope.go:117] "RemoveContainer" containerID="856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943"
Jul 29 22:12:25.837628 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.837609    2572 scope.go:117] "RemoveContainer" containerID="856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943"
Jul 29 22:12:25.837888 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:12:25.837868    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943\": container with ID starting with 856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943 not found: ID does not exist" containerID="856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943"
Jul 29 22:12:25.837954 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.837895    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943"} err="failed to get container status \"856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943\": rpc error: code = NotFound desc = could not find container \"856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943\": container with ID starting with 856b3b6f710c386fa62e3218023a27cf52c8300506f6f0126b6114e9177bd943 not found: ID does not exist"
Jul 29 22:12:25.850725 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.850690    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"]
Jul 29 22:12:25.854990 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.854961    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/auth-test-model-c9eccd-kserve-6b4498667-5hc24"]
Jul 29 22:12:25.957313 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:25.957268    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv" podUID="4fed3a72-695a-4f84-be34-453da65962fa" containerName="main" containerID="cri-o://8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2" gracePeriod=45
Jul 29 22:12:26.218513 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.218485    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"]
Jul 29 22:12:26.218909 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.218896    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" containerName="registry-server"
Jul 29 22:12:26.218909 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.218911    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" containerName="registry-server"
Jul 29 22:12:26.218994 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.218933    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19fdb909-6a79-481e-9dfa-44074b350e93" containerName="main"
Jul 29 22:12:26.218994 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.218939    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="19fdb909-6a79-481e-9dfa-44074b350e93" containerName="main"
Jul 29 22:12:26.219065 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.218998    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea8c2ef-d129-45ee-ad48-a0e6b86e03b5" containerName="registry-server"
Jul 29 22:12:26.219065 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.219007    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="19fdb909-6a79-481e-9dfa-44074b350e93" containerName="main"
Jul 29 22:12:26.222293 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.222272    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.224641 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.224619    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"openshift-service-ca.crt\""
Jul 29 22:12:26.225303 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.225286    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"default-dockercfg-gkfjv\""
Jul 29 22:12:26.225399 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.225287    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"sub-test-model-c9eccd-kserve-self-signed-certs\""
Jul 29 22:12:26.225399 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.225320    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c9eccd\"/\"kube-root-ca.crt\""
Jul 29 22:12:26.229491 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.229441    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:12:26.230754 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.230734    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"]
Jul 29 22:12:26.244759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.244729    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:26.251769 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.251746    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-tmp-dir\") pod \"4fed3a72-695a-4f84-be34-453da65962fa\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") "
Jul 29 22:12:26.251918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.251801    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-dshm\") pod \"4fed3a72-695a-4f84-be34-453da65962fa\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") "
Jul 29 22:12:26.251918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.251850    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-model-cache\") pod \"4fed3a72-695a-4f84-be34-453da65962fa\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") "
Jul 29 22:12:26.251918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.251876    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fed3a72-695a-4f84-be34-453da65962fa-tls-certs\") pod \"4fed3a72-695a-4f84-be34-453da65962fa\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") "
Jul 29 22:12:26.251918 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.251902    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-home\") pod \"4fed3a72-695a-4f84-be34-453da65962fa\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") "
Jul 29 22:12:26.252136 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.251956    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6t7p\" (UniqueName: \"kubernetes.io/projected/4fed3a72-695a-4f84-be34-453da65962fa-kube-api-access-x6t7p\") pod \"4fed3a72-695a-4f84-be34-453da65962fa\" (UID: \"4fed3a72-695a-4f84-be34-453da65962fa\") "
Jul 29 22:12:26.252136 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252049    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4fed3a72-695a-4f84-be34-453da65962fa" (UID: "4fed3a72-695a-4f84-be34-453da65962fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:26.252136 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-dshm\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.252339 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252134    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-model-cache\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.252339 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252155    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "4fed3a72-695a-4f84-be34-453da65962fa" (UID: "4fed3a72-695a-4f84-be34-453da65962fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:26.252339 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252210    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-home\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.252339 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252218    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-home" (OuterVolumeSpecName: "home") pod "4fed3a72-695a-4f84-be34-453da65962fa" (UID: "4fed3a72-695a-4f84-be34-453da65962fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:26.252339 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252324    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-tmp-dir\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.252601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252359    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6wsk\" (UniqueName: \"kubernetes.io/projected/7492d3fb-100f-46bc-8274-280fdfeb9b26-kube-api-access-t6wsk\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.252601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252379    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7492d3fb-100f-46bc-8274-280fdfeb9b26-tls-certs\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.252601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252515    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:26.252601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252532    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:26.252601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.252540    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:26.254197 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.254145    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fed3a72-695a-4f84-be34-453da65962fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4fed3a72-695a-4f84-be34-453da65962fa" (UID: "4fed3a72-695a-4f84-be34-453da65962fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:12:26.254295 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.254228    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-dshm" (OuterVolumeSpecName: "dshm") pod "4fed3a72-695a-4f84-be34-453da65962fa" (UID: "4fed3a72-695a-4f84-be34-453da65962fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:12:26.254295 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.254238    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fed3a72-695a-4f84-be34-453da65962fa-kube-api-access-x6t7p" (OuterVolumeSpecName: "kube-api-access-x6t7p") pod "4fed3a72-695a-4f84-be34-453da65962fa" (UID: "4fed3a72-695a-4f84-be34-453da65962fa"). InnerVolumeSpecName "kube-api-access-x6t7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:12:26.353388 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353298    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-model-cache\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353388 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353354    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-home\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353413    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-tmp-dir\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353435    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6wsk\" (UniqueName: \"kubernetes.io/projected/7492d3fb-100f-46bc-8274-280fdfeb9b26-kube-api-access-t6wsk\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353493    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7492d3fb-100f-46bc-8274-280fdfeb9b26-tls-certs\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353553    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-dshm\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353603    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6t7p\" (UniqueName: \"kubernetes.io/projected/4fed3a72-695a-4f84-be34-453da65962fa-kube-api-access-x6t7p\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353619    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fed3a72-695a-4f84-be34-453da65962fa-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:26.353640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353632    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fed3a72-695a-4f84-be34-453da65962fa-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:12:26.353969 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353742    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-model-cache\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353969 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353786    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-home\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.353969 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.353849    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-tmp-dir\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.355798 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.355772    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-dshm\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.356014 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.355994    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7492d3fb-100f-46bc-8274-280fdfeb9b26-tls-certs\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.361955 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.361933    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6wsk\" (UniqueName: \"kubernetes.io/projected/7492d3fb-100f-46bc-8274-280fdfeb9b26-kube-api-access-t6wsk\") pod \"sub-test-model-c9eccd-kserve-777d56999c-n2cvz\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") " pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.535377 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.535337    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:26.674008 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.673979    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"]
Jul 29 22:12:26.674772 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:12:26.674747    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7492d3fb_100f_46bc_8274_280fdfeb9b26.slice/crio-b03516b0629770b710eef19029443fa6f655290848f672d894a8bded4e414e22 WatchSource:0}: Error finding container b03516b0629770b710eef19029443fa6f655290848f672d894a8bded4e414e22: Status 404 returned error can't find the container with id b03516b0629770b710eef19029443fa6f655290848f672d894a8bded4e414e22
Jul 29 22:12:26.834474 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.834435    2572 generic.go:356] "Generic (PLEG): container finished" podID="4fed3a72-695a-4f84-be34-453da65962fa" containerID="8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2" exitCode=0
Jul 29 22:12:26.834684 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.834481    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"
Jul 29 22:12:26.834684 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.834525    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv" event={"ID":"4fed3a72-695a-4f84-be34-453da65962fa","Type":"ContainerDied","Data":"8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2"}
Jul 29 22:12:26.834684 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.834568    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv" event={"ID":"4fed3a72-695a-4f84-be34-453da65962fa","Type":"ContainerDied","Data":"fb83c2f89b33874da6abf142904f6c3d3b0025ea3b84a9384efd4937eb2ff912"}
Jul 29 22:12:26.834684 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.834590    2572 scope.go:117] "RemoveContainer" containerID="8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2"
Jul 29 22:12:26.836384 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.836357    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz" event={"ID":"7492d3fb-100f-46bc-8274-280fdfeb9b26","Type":"ContainerStarted","Data":"8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb"}
Jul 29 22:12:26.836523 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.836406    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz" event={"ID":"7492d3fb-100f-46bc-8274-280fdfeb9b26","Type":"ContainerStarted","Data":"b03516b0629770b710eef19029443fa6f655290848f672d894a8bded4e414e22"}
Jul 29 22:12:26.846447 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.846421    2572 scope.go:117] "RemoveContainer" containerID="8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2"
Jul 29 22:12:26.846720 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:12:26.846700    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2\": container with ID starting with 8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2 not found: ID does not exist" containerID="8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2"
Jul 29 22:12:26.846789 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.846727    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2"} err="failed to get container status \"8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2\": rpc error: code = NotFound desc = could not find container \"8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2\": container with ID starting with 8e78e4d8423d2db1bc86d50973e039ec0ff7112b4cc5cf3c024c67178f1dfee2 not found: ID does not exist"
Jul 29 22:12:26.856953 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.856906    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz" podStartSLOduration=0.856887516 podStartE2EDuration="856.887516ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:12:26.856503134 +0000 UTC m=+3358.073280596" watchObservedRunningTime="2026-07-29 22:12:26.856887516 +0000 UTC m=+3358.073664982"
Jul 29 22:12:26.872806 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.872737    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"]
Jul 29 22:12:26.881655 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:26.881623    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/auth-test-model-ca895e-kserve-67f99b8c7f-f47hv"]
Jul 29 22:12:27.305811 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:27.305773    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19fdb909-6a79-481e-9dfa-44074b350e93" path="/var/lib/kubelet/pods/19fdb909-6a79-481e-9dfa-44074b350e93/volumes"
Jul 29 22:12:27.306395 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:27.306375    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fed3a72-695a-4f84-be34-453da65962fa" path="/var/lib/kubelet/pods/4fed3a72-695a-4f84-be34-453da65962fa/volumes"
Jul 29 22:12:27.416631 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:27.416590    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:30.329151 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:30.329114    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:12:33.417335 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:33.417294    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:33.517875 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:33.517842    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:36.535967 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:36.535920    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:36.536513 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:36.535990    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:36.548709 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:36.548681    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:36.906339 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:36.906258    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:12:41.808624 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.808588    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"]
Jul 29 22:12:41.809293 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.809272    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fed3a72-695a-4f84-be34-453da65962fa" containerName="main"
Jul 29 22:12:41.809293 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.809294    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fed3a72-695a-4f84-be34-453da65962fa" containerName="main"
Jul 29 22:12:41.809466 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.809402    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fed3a72-695a-4f84-be34-453da65962fa" containerName="main"
Jul 29 22:12:41.812809 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.812789    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:41.815994 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.815972    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"sub-test-model-ca895e-kserve-self-signed-certs\""
Jul 29 22:12:41.816118 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.815991    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"openshift-service-ca.crt\""
Jul 29 22:12:41.816118 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.815996    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"default-dockercfg-hbjlz\""
Jul 29 22:12:41.816118 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.815980    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"kube-root-ca.crt\""
Jul 29 22:12:41.823306 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.823283    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"]
Jul 29 22:12:41.836304 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.836258    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:12:41.899861 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.899825    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-dshm\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:41.900038 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.899901    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-home\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:41.900038 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.899929    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-model-cache\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:41.900038 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.899983    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tls-certs\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:41.900038 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.900007    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tmp-dir\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:41.900210 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:41.900059    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmgxl\" (UniqueName: \"kubernetes.io/projected/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-kube-api-access-wmgxl\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001322 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001284    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-home\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001322 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001322    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-model-cache\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001587 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001352    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tls-certs\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001587 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001480    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tmp-dir\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001587 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001556    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmgxl\" (UniqueName: \"kubernetes.io/projected/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-kube-api-access-wmgxl\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001818 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001697    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-dshm\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001818 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001760    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-home\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001947 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001810    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-model-cache\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.001947 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.001868    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tmp-dir\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.003722 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.003696    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-dshm\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.003830 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.003776    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tls-certs\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.030379 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.030349    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmgxl\" (UniqueName: \"kubernetes.io/projected/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-kube-api-access-wmgxl\") pod \"sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") " pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.124264 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.124149    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:42.262971 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.262939    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"]
Jul 29 22:12:42.264995 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:12:42.264963    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3c3fc0b_eeb7_4978_b58c_1a9e34454bc2.slice/crio-1ef276a366df6406bbcb64331fc6971e569bda2a1d889911530a4dab7c0bd8f8 WatchSource:0}: Error finding container 1ef276a366df6406bbcb64331fc6971e569bda2a1d889911530a4dab7c0bd8f8: Status 404 returned error can't find the container with id 1ef276a366df6406bbcb64331fc6971e569bda2a1d889911530a4dab7c0bd8f8
Jul 29 22:12:42.931594 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.931554    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7" event={"ID":"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2","Type":"ContainerStarted","Data":"63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a"}
Jul 29 22:12:42.932091 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.931605    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7" event={"ID":"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2","Type":"ContainerStarted","Data":"1ef276a366df6406bbcb64331fc6971e569bda2a1d889911530a4dab7c0bd8f8"}
Jul 29 22:12:42.948748 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:42.948696    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7" podStartSLOduration=1.94867959 podStartE2EDuration="1.94867959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:12:42.947597962 +0000 UTC m=+3374.164375425" watchObservedRunningTime="2026-07-29 22:12:42.94867959 +0000 UTC m=+3374.165457052"
Jul 29 22:12:45.622906 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:45.622865    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:12:52.124524 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:52.124485    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:52.124524 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:52.124534    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:52.137157 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:52.137131    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:12:53.000579 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:12:53.000547    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:13:39.927120 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:13:39.927025    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:13:57.223350 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:13:57.223315    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:14:21.032906 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:21.032866    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:14:27.087922 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:27.087884    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"]
Jul 29 22:14:27.161403 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:27.161366    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:14:27.997639 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:27.997602    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"]
Jul 29 22:14:28.016229 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:28.016198    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:14:37.645880 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:37.645845    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:14:40.627622 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:40.627590    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:14:42.107587 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.107546    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz" podUID="7492d3fb-100f-46bc-8274-280fdfeb9b26" containerName="main" containerID="cri-o://8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb" gracePeriod=45
Jul 29 22:14:42.363383 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.363313    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:14:42.447574 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447538    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-home\") pod \"7492d3fb-100f-46bc-8274-280fdfeb9b26\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") "
Jul 29 22:14:42.447574 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447584    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-model-cache\") pod \"7492d3fb-100f-46bc-8274-280fdfeb9b26\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") "
Jul 29 22:14:42.447832 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447626    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7492d3fb-100f-46bc-8274-280fdfeb9b26-tls-certs\") pod \"7492d3fb-100f-46bc-8274-280fdfeb9b26\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") "
Jul 29 22:14:42.447832 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447645    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-tmp-dir\") pod \"7492d3fb-100f-46bc-8274-280fdfeb9b26\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") "
Jul 29 22:14:42.447832 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447695    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-dshm\") pod \"7492d3fb-100f-46bc-8274-280fdfeb9b26\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") "
Jul 29 22:14:42.447964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447859    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6wsk\" (UniqueName: \"kubernetes.io/projected/7492d3fb-100f-46bc-8274-280fdfeb9b26-kube-api-access-t6wsk\") pod \"7492d3fb-100f-46bc-8274-280fdfeb9b26\" (UID: \"7492d3fb-100f-46bc-8274-280fdfeb9b26\") "
Jul 29 22:14:42.447964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447856    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-home" (OuterVolumeSpecName: "home") pod "7492d3fb-100f-46bc-8274-280fdfeb9b26" (UID: "7492d3fb-100f-46bc-8274-280fdfeb9b26"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:42.447964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.447915    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-model-cache" (OuterVolumeSpecName: "model-cache") pod "7492d3fb-100f-46bc-8274-280fdfeb9b26" (UID: "7492d3fb-100f-46bc-8274-280fdfeb9b26"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:42.448077 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.448028    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7492d3fb-100f-46bc-8274-280fdfeb9b26" (UID: "7492d3fb-100f-46bc-8274-280fdfeb9b26"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:42.448347 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.448329    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:42.448438 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.448349    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:42.448438 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.448360    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:42.449882 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.449860    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-dshm" (OuterVolumeSpecName: "dshm") pod "7492d3fb-100f-46bc-8274-280fdfeb9b26" (UID: "7492d3fb-100f-46bc-8274-280fdfeb9b26"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:42.449982 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.449891    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7492d3fb-100f-46bc-8274-280fdfeb9b26-kube-api-access-t6wsk" (OuterVolumeSpecName: "kube-api-access-t6wsk") pod "7492d3fb-100f-46bc-8274-280fdfeb9b26" (UID: "7492d3fb-100f-46bc-8274-280fdfeb9b26"). InnerVolumeSpecName "kube-api-access-t6wsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:14:42.449982 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.449909    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7492d3fb-100f-46bc-8274-280fdfeb9b26-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7492d3fb-100f-46bc-8274-280fdfeb9b26" (UID: "7492d3fb-100f-46bc-8274-280fdfeb9b26"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:14:42.549597 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.549554    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7492d3fb-100f-46bc-8274-280fdfeb9b26-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:42.549597 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.549595    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7492d3fb-100f-46bc-8274-280fdfeb9b26-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:42.549597 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.549606    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6wsk\" (UniqueName: \"kubernetes.io/projected/7492d3fb-100f-46bc-8274-280fdfeb9b26-kube-api-access-t6wsk\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:42.615839 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.615735    2572 generic.go:356] "Generic (PLEG): container finished" podID="7492d3fb-100f-46bc-8274-280fdfeb9b26" containerID="8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb" exitCode=0
Jul 29 22:14:42.615839 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.615787    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"
Jul 29 22:14:42.615839 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.615827    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz" event={"ID":"7492d3fb-100f-46bc-8274-280fdfeb9b26","Type":"ContainerDied","Data":"8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb"}
Jul 29 22:14:42.616121 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.615877    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz" event={"ID":"7492d3fb-100f-46bc-8274-280fdfeb9b26","Type":"ContainerDied","Data":"b03516b0629770b710eef19029443fa6f655290848f672d894a8bded4e414e22"}
Jul 29 22:14:42.616121 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.615896    2572 scope.go:117] "RemoveContainer" containerID="8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb"
Jul 29 22:14:42.625276 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.625262    2572 scope.go:117] "RemoveContainer" containerID="8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb"
Jul 29 22:14:42.625549 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:14:42.625530    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb\": container with ID starting with 8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb not found: ID does not exist" containerID="8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb"
Jul 29 22:14:42.625600 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.625557    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb"} err="failed to get container status \"8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb\": rpc error: code = NotFound desc = could not find container \"8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb\": container with ID starting with 8e65f8190f99674db95de0215de2f923e8f434695fa7a62a90a7ac561bb59eeb not found: ID does not exist"
Jul 29 22:14:42.636815 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.636789    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"]
Jul 29 22:14:42.641385 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:42.641360    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c9eccd/sub-test-model-c9eccd-kserve-777d56999c-n2cvz"]
Jul 29 22:14:43.017091 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.017044    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7" podUID="d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" containerName="main" containerID="cri-o://63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a" gracePeriod=45
Jul 29 22:14:43.282681 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.282610    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:14:43.311704 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.311662    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7492d3fb-100f-46bc-8274-280fdfeb9b26" path="/var/lib/kubelet/pods/7492d3fb-100f-46bc-8274-280fdfeb9b26/volumes"
Jul 29 22:14:43.356691 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356645    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-home\") pod \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") "
Jul 29 22:14:43.356691 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356683    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tls-certs\") pod \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") "
Jul 29 22:14:43.356955 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356718    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-dshm\") pod \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") "
Jul 29 22:14:43.356955 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356779    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tmp-dir\") pod \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") "
Jul 29 22:14:43.356955 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356831    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-model-cache\") pod \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") "
Jul 29 22:14:43.356955 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356877    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmgxl\" (UniqueName: \"kubernetes.io/projected/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-kube-api-access-wmgxl\") pod \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\" (UID: \"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2\") "
Jul 29 22:14:43.357165 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.356978    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-home" (OuterVolumeSpecName: "home") pod "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" (UID: "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:43.357165 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.357148    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-model-cache" (OuterVolumeSpecName: "model-cache") pod "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" (UID: "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:43.357298 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.357186    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" (UID: "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:43.357298 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.357285    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:43.357397 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.357316    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:43.357397 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.357327    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:43.359139 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.359111    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-kube-api-access-wmgxl" (OuterVolumeSpecName: "kube-api-access-wmgxl") pod "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" (UID: "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2"). InnerVolumeSpecName "kube-api-access-wmgxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:14:43.359391 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.359361    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-dshm" (OuterVolumeSpecName: "dshm") pod "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" (UID: "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:14:43.359524 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.359409    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" (UID: "d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:14:43.457790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.457748    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmgxl\" (UniqueName: \"kubernetes.io/projected/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-kube-api-access-wmgxl\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:43.457790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.457785    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:43.457790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.457796    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:14:43.623976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.623883    2572 generic.go:356] "Generic (PLEG): container finished" podID="d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" containerID="63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a" exitCode=0
Jul 29 22:14:43.623976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.623935    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7" event={"ID":"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2","Type":"ContainerDied","Data":"63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a"}
Jul 29 22:14:43.623976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.623950    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"
Jul 29 22:14:43.623976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.623976    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7" event={"ID":"d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2","Type":"ContainerDied","Data":"1ef276a366df6406bbcb64331fc6971e569bda2a1d889911530a4dab7c0bd8f8"}
Jul 29 22:14:43.624278 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.623995    2572 scope.go:117] "RemoveContainer" containerID="63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a"
Jul 29 22:14:43.634691 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.634669    2572 scope.go:117] "RemoveContainer" containerID="63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a"
Jul 29 22:14:43.635038 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:14:43.635018    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a\": container with ID starting with 63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a not found: ID does not exist" containerID="63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a"
Jul 29 22:14:43.635084 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.635048    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a"} err="failed to get container status \"63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a\": rpc error: code = NotFound desc = could not find container \"63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a\": container with ID starting with 63f9c1de70ee7ef2a645d60335b735d3a9c151b16fcaa5c5ff7277eb95bcd29a not found: ID does not exist"
Jul 29 22:14:43.647992 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.647960    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"]
Jul 29 22:14:43.653585 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:43.653557    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/sub-test-model-ca895e-kserve-84cb4d9dcc-69dx7"]
Jul 29 22:14:45.304788 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:14:45.304754    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" path="/var/lib/kubelet/pods/d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2/volumes"
Jul 29 22:15:00.140152 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140068    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"]
Jul 29 22:15:00.140652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140523    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" containerName="main"
Jul 29 22:15:00.140652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140536    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" containerName="main"
Jul 29 22:15:00.140652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140558    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7492d3fb-100f-46bc-8274-280fdfeb9b26" containerName="main"
Jul 29 22:15:00.140652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140564    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7492d3fb-100f-46bc-8274-280fdfeb9b26" containerName="main"
Jul 29 22:15:00.140652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140628    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c3fc0b-eeb7-4978-b58c-1a9e34454bc2" containerName="main"
Jul 29 22:15:00.140652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.140638    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7492d3fb-100f-46bc-8274-280fdfeb9b26" containerName="main"
Jul 29 22:15:00.145315 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.145294    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"]
Jul 29 22:15:00.145457 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.145435    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"
Jul 29 22:15:00.148149 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.148125    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxbps\""
Jul 29 22:15:00.148679 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.148663    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"]
Jul 29 22:15:00.148777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.148760    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"
Jul 29 22:15:00.152016 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.151987    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"
Jul 29 22:15:00.153438 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.153409    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"]
Jul 29 22:15:00.158981 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.158954    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"]
Jul 29 22:15:00.160513 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.160490    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"
Jul 29 22:15:00.162575 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.162549    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"]
Jul 29 22:15:00.168963 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.168945    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"
Jul 29 22:15:00.175793 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.175773    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"
Jul 29 22:15:00.335559 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.335533    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"]
Jul 29 22:15:00.338436 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:15:00.338394    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod236bba51_a700_4bf9_87bb_8c29e7c526eb.slice/crio-a697e599bbd47d0ec6c899ad27a0eb1711ad9340c77138ab8f1eef5846720c76 WatchSource:0}: Error finding container a697e599bbd47d0ec6c899ad27a0eb1711ad9340c77138ab8f1eef5846720c76: Status 404 returned error can't find the container with id a697e599bbd47d0ec6c899ad27a0eb1711ad9340c77138ab8f1eef5846720c76
Jul 29 22:15:00.561602 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.561577    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"]
Jul 29 22:15:00.562985 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:15:00.562958    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae16bbba_e25d_4bd3_ab19_62c77d9eaeb2.slice/crio-2ca130cb46f592f4450cb0c42bbef2c383ed2d65880cbce7d7d28b4aaac7a652 WatchSource:0}: Error finding container 2ca130cb46f592f4450cb0c42bbef2c383ed2d65880cbce7d7d28b4aaac7a652: Status 404 returned error can't find the container with id 2ca130cb46f592f4450cb0c42bbef2c383ed2d65880cbce7d7d28b4aaac7a652
Jul 29 22:15:00.591298 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.591276    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"]
Jul 29 22:15:00.592991 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:15:00.592961    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36eb5a73_5887_4adc_bd22_0fe4690a9de3.slice/crio-00d27113794cef804cebc601edbc59a5bdf35b1f977780ae8a8e7a1cd99a32ac WatchSource:0}: Error finding container 00d27113794cef804cebc601edbc59a5bdf35b1f977780ae8a8e7a1cd99a32ac: Status 404 returned error can't find the container with id 00d27113794cef804cebc601edbc59a5bdf35b1f977780ae8a8e7a1cd99a32ac
Jul 29 22:15:00.719046 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.719007    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw" event={"ID":"36eb5a73-5887-4adc-bd22-0fe4690a9de3","Type":"ContainerStarted","Data":"00d27113794cef804cebc601edbc59a5bdf35b1f977780ae8a8e7a1cd99a32ac"}
Jul 29 22:15:00.720190 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.720137    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22" event={"ID":"ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2","Type":"ContainerStarted","Data":"2ca130cb46f592f4450cb0c42bbef2c383ed2d65880cbce7d7d28b4aaac7a652"}
Jul 29 22:15:00.721688 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.721665    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l" event={"ID":"236bba51-a700-4bf9-87bb-8c29e7c526eb","Type":"ContainerStarted","Data":"802abe5a4bb95fc300fe1fa5250848452ce96423b7959ea974dd33b06690d2d1"}
Jul 29 22:15:00.721809 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.721718    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l" event={"ID":"236bba51-a700-4bf9-87bb-8c29e7c526eb","Type":"ContainerStarted","Data":"a697e599bbd47d0ec6c899ad27a0eb1711ad9340c77138ab8f1eef5846720c76"}
Jul 29 22:15:00.737357 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:00.737312    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l" podStartSLOduration=0.737299228 podStartE2EDuration="737.299228ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:15:00.736305661 +0000 UTC m=+3511.953083122" watchObservedRunningTime="2026-07-29 22:15:00.737299228 +0000 UTC m=+3511.954076712"
Jul 29 22:15:01.728713 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:01.728680    2572 generic.go:356] "Generic (PLEG): container finished" podID="36eb5a73-5887-4adc-bd22-0fe4690a9de3" containerID="f14cfe4a3f0ac059b3aa951f17313d53fed2db10755de6e50a34e18ff07ccd76" exitCode=0
Jul 29 22:15:01.729074 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:01.728763    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw" event={"ID":"36eb5a73-5887-4adc-bd22-0fe4690a9de3","Type":"ContainerDied","Data":"f14cfe4a3f0ac059b3aa951f17313d53fed2db10755de6e50a34e18ff07ccd76"}
Jul 29 22:15:01.730285 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:01.730262    2572 generic.go:356] "Generic (PLEG): container finished" podID="ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2" containerID="2ec1763b59d579afe99469c79923c7ee1b344b90215d3b3b9b981596f0394eb7" exitCode=0
Jul 29 22:15:01.730385 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:01.730323    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22" event={"ID":"ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2","Type":"ContainerDied","Data":"2ec1763b59d579afe99469c79923c7ee1b344b90215d3b3b9b981596f0394eb7"}
Jul 29 22:15:01.731669 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:01.731648    2572 generic.go:356] "Generic (PLEG): container finished" podID="236bba51-a700-4bf9-87bb-8c29e7c526eb" containerID="802abe5a4bb95fc300fe1fa5250848452ce96423b7959ea974dd33b06690d2d1" exitCode=0
Jul 29 22:15:01.731747 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:01.731711    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l" event={"ID":"236bba51-a700-4bf9-87bb-8c29e7c526eb","Type":"ContainerDied","Data":"802abe5a4bb95fc300fe1fa5250848452ce96423b7959ea974dd33b06690d2d1"}
Jul 29 22:15:02.883859 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:02.883817    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"
Jul 29 22:15:02.924757 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:02.924728    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"
Jul 29 22:15:02.928273 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:02.928251    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"
Jul 29 22:15:03.743706 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.743672    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"
Jul 29 22:15:03.743706 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.743693    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw" event={"ID":"36eb5a73-5887-4adc-bd22-0fe4690a9de3","Type":"ContainerDied","Data":"00d27113794cef804cebc601edbc59a5bdf35b1f977780ae8a8e7a1cd99a32ac"}
Jul 29 22:15:03.743938 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.743742    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00d27113794cef804cebc601edbc59a5bdf35b1f977780ae8a8e7a1cd99a32ac"
Jul 29 22:15:03.744990 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.744966    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22" event={"ID":"ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2","Type":"ContainerDied","Data":"2ca130cb46f592f4450cb0c42bbef2c383ed2d65880cbce7d7d28b4aaac7a652"}
Jul 29 22:15:03.745105 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.745004    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ca130cb46f592f4450cb0c42bbef2c383ed2d65880cbce7d7d28b4aaac7a652"
Jul 29 22:15:03.745105 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.744973    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"
Jul 29 22:15:03.746304 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.746287    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l"
Jul 29 22:15:03.746386 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.746332    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29756055-2gq5l" event={"ID":"236bba51-a700-4bf9-87bb-8c29e7c526eb","Type":"ContainerDied","Data":"a697e599bbd47d0ec6c899ad27a0eb1711ad9340c77138ab8f1eef5846720c76"}
Jul 29 22:15:03.746386 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:03.746369    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a697e599bbd47d0ec6c899ad27a0eb1711ad9340c77138ab8f1eef5846720c76"
Jul 29 22:15:11.410697 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.410660    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"]
Jul 29 22:15:11.411127 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411113    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="236bba51-a700-4bf9-87bb-8c29e7c526eb" containerName="cleanup"
Jul 29 22:15:11.411239 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411128    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="236bba51-a700-4bf9-87bb-8c29e7c526eb" containerName="cleanup"
Jul 29 22:15:11.411239 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411137    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2" containerName="cleanup"
Jul 29 22:15:11.411239 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411143    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2" containerName="cleanup"
Jul 29 22:15:11.411239 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411161    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36eb5a73-5887-4adc-bd22-0fe4690a9de3" containerName="cleanup"
Jul 29 22:15:11.411239 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411191    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="36eb5a73-5887-4adc-bd22-0fe4690a9de3" containerName="cleanup"
Jul 29 22:15:11.411390 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411260    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2" containerName="cleanup"
Jul 29 22:15:11.411390 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411270    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="236bba51-a700-4bf9-87bb-8c29e7c526eb" containerName="cleanup"
Jul 29 22:15:11.411390 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.411276    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="36eb5a73-5887-4adc-bd22-0fe4690a9de3" containerName="cleanup"
Jul 29 22:15:11.414512 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.414491    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.417011 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.416986    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"openshift-service-ca.crt\""
Jul 29 22:15:11.417110 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.416991    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"kube-root-ca.crt\""
Jul 29 22:15:11.417815 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.417784    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"default-dockercfg-hbjlz\""
Jul 29 22:15:11.418060 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.417950    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-ca895e\"/\"rate-test-model-ca895e-kserve-self-signed-certs\""
Jul 29 22:15:11.423369 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.423345    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"]
Jul 29 22:15:11.440714 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.440687    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:15:11.516846 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.516814    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e36b02-8eba-4dba-b004-36f2f443e8e4-tls-certs\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.516846 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.516850    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-tmp-dir\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.517073 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.516868    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-home\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.517073 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.516988    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-dshm\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.517073 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.517025    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2kdp\" (UniqueName: \"kubernetes.io/projected/92e36b02-8eba-4dba-b004-36f2f443e8e4-kube-api-access-g2kdp\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.517221 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.517101    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-model-cache\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618320 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618272    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-model-cache\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618491 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618341    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e36b02-8eba-4dba-b004-36f2f443e8e4-tls-certs\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618491 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618396    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-tmp-dir\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618491 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618441    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-home\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618623 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618517    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-dshm\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618623 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618555    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2kdp\" (UniqueName: \"kubernetes.io/projected/92e36b02-8eba-4dba-b004-36f2f443e8e4-kube-api-access-g2kdp\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618886 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618862    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-tmp-dir\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618951 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618884    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-home\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.618951 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.618939    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-model-cache\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.620887 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.620846    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-dshm\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.621041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.621026    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e36b02-8eba-4dba-b004-36f2f443e8e4-tls-certs\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.627909 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.627874    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2kdp\" (UniqueName: \"kubernetes.io/projected/92e36b02-8eba-4dba-b004-36f2f443e8e4-kube-api-access-g2kdp\") pod \"rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") " pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.725851 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.725806    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:11.869302 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:11.869273    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"]
Jul 29 22:15:11.871718 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:15:11.871689    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod92e36b02_8eba_4dba_b004_36f2f443e8e4.slice/crio-f50a7e4e9d29264ea8de0751cdad68a5428c7d48c4ea02b72a1bfced0a2745ca WatchSource:0}: Error finding container f50a7e4e9d29264ea8de0751cdad68a5428c7d48c4ea02b72a1bfced0a2745ca: Status 404 returned error can't find the container with id f50a7e4e9d29264ea8de0751cdad68a5428c7d48c4ea02b72a1bfced0a2745ca
Jul 29 22:15:12.799523 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:12.799482    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn" event={"ID":"92e36b02-8eba-4dba-b004-36f2f443e8e4","Type":"ContainerStarted","Data":"472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8"}
Jul 29 22:15:12.799900 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:12.799535    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn" event={"ID":"92e36b02-8eba-4dba-b004-36f2f443e8e4","Type":"ContainerStarted","Data":"f50a7e4e9d29264ea8de0751cdad68a5428c7d48c4ea02b72a1bfced0a2745ca"}
Jul 29 22:15:12.818821 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:12.818778    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn" podStartSLOduration=1.8187646069999999 podStartE2EDuration="1.818764607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:15:12.816703848 +0000 UTC m=+3524.033481310" watchObservedRunningTime="2026-07-29 22:15:12.818764607 +0000 UTC m=+3524.035542068"
Jul 29 22:15:14.928428 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:14.928388    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:15:21.726736 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:21.726700    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:21.727151 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:21.726779    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:21.739212 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:21.739185    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:21.860710 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:21.860677    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:15:54.117901 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:15:54.117849    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:16:02.327202 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:02.327148    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:16:23.229588 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:23.229493    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:16:45.459309 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:45.459274    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:16:52.850708 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:52.850677    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:16:53.794150 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:53.794115    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"]
Jul 29 22:16:53.815826 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:53.815779    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:16:54.829402 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:54.829363    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7928-140115462261168"]
Jul 29 22:16:54.833063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:54.833044    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:54.835978 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:54.835952    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-chjbj\""
Jul 29 22:16:54.838827 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:54.838801    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462261168"]
Jul 29 22:16:54.900395 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:54.900356    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwxc2\" (UniqueName: \"kubernetes.io/projected/b939c90a-27d3-4d2b-8b30-617daaca4e11-kube-api-access-cwxc2\") pod \"test-curl-7928-140115462261168\" (UID: \"b939c90a-27d3-4d2b-8b30-617daaca4e11\") " pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:55.001081 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:55.001032    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwxc2\" (UniqueName: \"kubernetes.io/projected/b939c90a-27d3-4d2b-8b30-617daaca4e11-kube-api-access-cwxc2\") pod \"test-curl-7928-140115462261168\" (UID: \"b939c90a-27d3-4d2b-8b30-617daaca4e11\") " pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:55.009697 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:55.009662    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwxc2\" (UniqueName: \"kubernetes.io/projected/b939c90a-27d3-4d2b-8b30-617daaca4e11-kube-api-access-cwxc2\") pod \"test-curl-7928-140115462261168\" (UID: \"b939c90a-27d3-4d2b-8b30-617daaca4e11\") " pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:55.144256 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:55.144140    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:55.288539 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:55.288497    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462261168"]
Jul 29 22:16:55.290077 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:16:55.290053    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb939c90a_27d3_4d2b_8b30_617daaca4e11.slice/crio-0d7ec4d341dd15c1c0d489ea5f7669637540908d10c63c7d50416ddafc232d13 WatchSource:0}: Error finding container 0d7ec4d341dd15c1c0d489ea5f7669637540908d10c63c7d50416ddafc232d13: Status 404 returned error can't find the container with id 0d7ec4d341dd15c1c0d489ea5f7669637540908d10c63c7d50416ddafc232d13
Jul 29 22:16:55.291928 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:55.291906    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 22:16:55.392619 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:55.392588    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462261168" event={"ID":"b939c90a-27d3-4d2b-8b30-617daaca4e11","Type":"ContainerStarted","Data":"0d7ec4d341dd15c1c0d489ea5f7669637540908d10c63c7d50416ddafc232d13"}
Jul 29 22:16:56.400601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:56.400503    2572 generic.go:356] "Generic (PLEG): container finished" podID="b939c90a-27d3-4d2b-8b30-617daaca4e11" containerID="90661e0795ab6f2086e01b1940ac8ec1b2b60496a119f7b629f4be86b869aee8" exitCode=0
Jul 29 22:16:56.400601 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:56.400577    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462261168" event={"ID":"b939c90a-27d3-4d2b-8b30-617daaca4e11","Type":"ContainerDied","Data":"90661e0795ab6f2086e01b1940ac8ec1b2b60496a119f7b629f4be86b869aee8"}
Jul 29 22:16:57.558067 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.558039    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:57.579095 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.579067    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7928-140115462261168_b939c90a-27d3-4d2b-8b30-617daaca4e11/test-curl-7928-140115462261168/0.log"
Jul 29 22:16:57.603555 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.603518    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7928-140115462261168"]
Jul 29 22:16:57.605911 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.605887    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7928-140115462261168"]
Jul 29 22:16:57.628855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.628820    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwxc2\" (UniqueName: \"kubernetes.io/projected/b939c90a-27d3-4d2b-8b30-617daaca4e11-kube-api-access-cwxc2\") pod \"b939c90a-27d3-4d2b-8b30-617daaca4e11\" (UID: \"b939c90a-27d3-4d2b-8b30-617daaca4e11\") "
Jul 29 22:16:57.631030 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.631001    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b939c90a-27d3-4d2b-8b30-617daaca4e11-kube-api-access-cwxc2" (OuterVolumeSpecName: "kube-api-access-cwxc2") pod "b939c90a-27d3-4d2b-8b30-617daaca4e11" (UID: "b939c90a-27d3-4d2b-8b30-617daaca4e11"). InnerVolumeSpecName "kube-api-access-cwxc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:16:57.729915 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.729881    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwxc2\" (UniqueName: \"kubernetes.io/projected/b939c90a-27d3-4d2b-8b30-617daaca4e11-kube-api-access-cwxc2\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:16:57.734545 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.734514    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7928-140115462226224"]
Jul 29 22:16:57.734980 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.734964    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b939c90a-27d3-4d2b-8b30-617daaca4e11" containerName="test-curl-7928-140115462261168"
Jul 29 22:16:57.735033 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.734982    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b939c90a-27d3-4d2b-8b30-617daaca4e11" containerName="test-curl-7928-140115462261168"
Jul 29 22:16:57.735092 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.735080    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b939c90a-27d3-4d2b-8b30-617daaca4e11" containerName="test-curl-7928-140115462261168"
Jul 29 22:16:57.738371 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.738351    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:16:57.744634 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.744608    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462226224"]
Jul 29 22:16:57.830619 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.830584    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrwtq\" (UniqueName: \"kubernetes.io/projected/2468a8d0-55c9-4b4f-be0b-e4c153c9109d-kube-api-access-zrwtq\") pod \"test-curl-7928-140115462226224\" (UID: \"2468a8d0-55c9-4b4f-be0b-e4c153c9109d\") " pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:16:57.931584 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.931544    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrwtq\" (UniqueName: \"kubernetes.io/projected/2468a8d0-55c9-4b4f-be0b-e4c153c9109d-kube-api-access-zrwtq\") pod \"test-curl-7928-140115462226224\" (UID: \"2468a8d0-55c9-4b4f-be0b-e4c153c9109d\") " pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:16:57.940109 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:57.940066    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrwtq\" (UniqueName: \"kubernetes.io/projected/2468a8d0-55c9-4b4f-be0b-e4c153c9109d-kube-api-access-zrwtq\") pod \"test-curl-7928-140115462226224\" (UID: \"2468a8d0-55c9-4b4f-be0b-e4c153c9109d\") " pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:16:58.049839 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:58.049747    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:16:58.191972 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:58.191939    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462226224"]
Jul 29 22:16:58.192089 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:16:58.192033    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2468a8d0_55c9_4b4f_be0b_e4c153c9109d.slice/crio-874b32a795fbeaf1a462defefe09a87a2812c7cd57becb9f83b6ed2d2af192b9 WatchSource:0}: Error finding container 874b32a795fbeaf1a462defefe09a87a2812c7cd57becb9f83b6ed2d2af192b9: Status 404 returned error can't find the container with id 874b32a795fbeaf1a462defefe09a87a2812c7cd57becb9f83b6ed2d2af192b9
Jul 29 22:16:58.413807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:58.413130    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d7ec4d341dd15c1c0d489ea5f7669637540908d10c63c7d50416ddafc232d13"
Jul 29 22:16:58.413807 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:58.413129    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:58.414932 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:58.414901    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462226224" event={"ID":"2468a8d0-55c9-4b4f-be0b-e4c153c9109d","Type":"ContainerStarted","Data":"874b32a795fbeaf1a462defefe09a87a2812c7cd57becb9f83b6ed2d2af192b9"}
Jul 29 22:16:58.424149 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:16:58.424113    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7928-140115462261168\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b939c90a-27d3-4d2b-8b30-617daaca4e11" pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:59.302971 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:16:59.302924    2572 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7928-140115462261168\" is forbidden: User \"system:node:ip-10-0-129-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-242.ec2.internal' and this object" podUID="b939c90a-27d3-4d2b-8b30-617daaca4e11" pod="opendatahub/test-curl-7928-140115462261168"
Jul 29 22:16:59.305362 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:59.305330    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b939c90a-27d3-4d2b-8b30-617daaca4e11" path="/var/lib/kubelet/pods/b939c90a-27d3-4d2b-8b30-617daaca4e11/volumes"
Jul 29 22:16:59.422690 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:59.422654    2572 generic.go:356] "Generic (PLEG): container finished" podID="2468a8d0-55c9-4b4f-be0b-e4c153c9109d" containerID="2da8fca77f03070a271628e0a4115780c79735395f6d90e90de2dd42f4031910" exitCode=0
Jul 29 22:16:59.422882 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:59.422736    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462226224" event={"ID":"2468a8d0-55c9-4b4f-be0b-e4c153c9109d","Type":"ContainerDied","Data":"2da8fca77f03070a271628e0a4115780c79735395f6d90e90de2dd42f4031910"}
Jul 29 22:16:59.727076 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:16:59.727029    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:17:00.568185 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.568142    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:17:00.590535 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.590503    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7928-140115462226224_2468a8d0-55c9-4b4f-be0b-e4c153c9109d/test-curl-7928-140115462226224/0.log"
Jul 29 22:17:00.612632 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.612596    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7928-140115462226224"]
Jul 29 22:17:00.614869 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.614848    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7928-140115462226224"]
Jul 29 22:17:00.658530 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.658500    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrwtq\" (UniqueName: \"kubernetes.io/projected/2468a8d0-55c9-4b4f-be0b-e4c153c9109d-kube-api-access-zrwtq\") pod \"2468a8d0-55c9-4b4f-be0b-e4c153c9109d\" (UID: \"2468a8d0-55c9-4b4f-be0b-e4c153c9109d\") "
Jul 29 22:17:00.660553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.660533    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2468a8d0-55c9-4b4f-be0b-e4c153c9109d-kube-api-access-zrwtq" (OuterVolumeSpecName: "kube-api-access-zrwtq") pod "2468a8d0-55c9-4b4f-be0b-e4c153c9109d" (UID: "2468a8d0-55c9-4b4f-be0b-e4c153c9109d"). InnerVolumeSpecName "kube-api-access-zrwtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:17:00.759392 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.759309    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrwtq\" (UniqueName: \"kubernetes.io/projected/2468a8d0-55c9-4b4f-be0b-e4c153c9109d-kube-api-access-zrwtq\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:00.784304 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.784270    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:00.784685 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.784670    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2468a8d0-55c9-4b4f-be0b-e4c153c9109d" containerName="test-curl-7928-140115462226224"
Jul 29 22:17:00.784685 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.784685    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="2468a8d0-55c9-4b4f-be0b-e4c153c9109d" containerName="test-curl-7928-140115462226224"
Jul 29 22:17:00.784812 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.784769    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="2468a8d0-55c9-4b4f-be0b-e4c153c9109d" containerName="test-curl-7928-140115462226224"
Jul 29 22:17:00.787792 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.787773    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:00.792278 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.792249    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:00.860769 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.860731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4zpg\" (UniqueName: \"kubernetes.io/projected/d632e1cb-3b95-4e68-a64a-e6347c183782-kube-api-access-z4zpg\") pod \"test-curl-7928-140115462225040\" (UID: \"d632e1cb-3b95-4e68-a64a-e6347c183782\") " pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:00.961311 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.961277    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4zpg\" (UniqueName: \"kubernetes.io/projected/d632e1cb-3b95-4e68-a64a-e6347c183782-kube-api-access-z4zpg\") pod \"test-curl-7928-140115462225040\" (UID: \"d632e1cb-3b95-4e68-a64a-e6347c183782\") " pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:00.970589 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:00.970559    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4zpg\" (UniqueName: \"kubernetes.io/projected/d632e1cb-3b95-4e68-a64a-e6347c183782-kube-api-access-z4zpg\") pod \"test-curl-7928-140115462225040\" (UID: \"d632e1cb-3b95-4e68-a64a-e6347c183782\") " pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:01.098860 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:01.098773    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:01.235504 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:01.235470    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:01.236883 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:17:01.236853    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd632e1cb_3b95_4e68_a64a_e6347c183782.slice/crio-287153e2a99a9122dac50c8bee4244281cdd67ffb3c5c1259461a8d49154e0b7 WatchSource:0}: Error finding container 287153e2a99a9122dac50c8bee4244281cdd67ffb3c5c1259461a8d49154e0b7: Status 404 returned error can't find the container with id 287153e2a99a9122dac50c8bee4244281cdd67ffb3c5c1259461a8d49154e0b7
Jul 29 22:17:01.305524 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:01.305499    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2468a8d0-55c9-4b4f-be0b-e4c153c9109d" path="/var/lib/kubelet/pods/2468a8d0-55c9-4b4f-be0b-e4c153c9109d/volumes"
Jul 29 22:17:01.436665 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:01.436626    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462225040" event={"ID":"d632e1cb-3b95-4e68-a64a-e6347c183782","Type":"ContainerStarted","Data":"287153e2a99a9122dac50c8bee4244281cdd67ffb3c5c1259461a8d49154e0b7"}
Jul 29 22:17:01.437935 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:01.437917    2572 scope.go:117] "RemoveContainer" containerID="2da8fca77f03070a271628e0a4115780c79735395f6d90e90de2dd42f4031910"
Jul 29 22:17:01.438064 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:01.437917    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462226224"
Jul 29 22:17:02.446065 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:02.446027    2572 generic.go:356] "Generic (PLEG): container finished" podID="d632e1cb-3b95-4e68-a64a-e6347c183782" containerID="4fb518c302dc6133130b4a70eba986ba1f7333addb855970b141f2e692a0df85" exitCode=0
Jul 29 22:17:02.446530 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:02.446110    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462225040" event={"ID":"d632e1cb-3b95-4e68-a64a-e6347c183782","Type":"ContainerDied","Data":"4fb518c302dc6133130b4a70eba986ba1f7333addb855970b141f2e692a0df85"}
Jul 29 22:17:03.610974 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.610944    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:03.633879 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.633839    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7928-140115462225040_d632e1cb-3b95-4e68-a64a-e6347c183782/test-curl-7928-140115462225040/0.log"
Jul 29 22:17:03.658468 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.658431    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:03.662032 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.662000    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:03.688654 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.688623    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4zpg\" (UniqueName: \"kubernetes.io/projected/d632e1cb-3b95-4e68-a64a-e6347c183782-kube-api-access-z4zpg\") pod \"d632e1cb-3b95-4e68-a64a-e6347c183782\" (UID: \"d632e1cb-3b95-4e68-a64a-e6347c183782\") "
Jul 29 22:17:03.690631 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.690602    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d632e1cb-3b95-4e68-a64a-e6347c183782-kube-api-access-z4zpg" (OuterVolumeSpecName: "kube-api-access-z4zpg") pod "d632e1cb-3b95-4e68-a64a-e6347c183782" (UID: "d632e1cb-3b95-4e68-a64a-e6347c183782"). InnerVolumeSpecName "kube-api-access-z4zpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:17:03.784405 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.784309    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:03.784785 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.784770    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d632e1cb-3b95-4e68-a64a-e6347c183782" containerName="test-curl-7928-140115462225040"
Jul 29 22:17:03.784833 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.784787    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d632e1cb-3b95-4e68-a64a-e6347c183782" containerName="test-curl-7928-140115462225040"
Jul 29 22:17:03.784887 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.784875    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="d632e1cb-3b95-4e68-a64a-e6347c183782" containerName="test-curl-7928-140115462225040"
Jul 29 22:17:03.788053 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.788035    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:03.789939 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.789911    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4zpg\" (UniqueName: \"kubernetes.io/projected/d632e1cb-3b95-4e68-a64a-e6347c183782-kube-api-access-z4zpg\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:03.795560 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.795522    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:03.890608 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.890572    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlh8j\" (UniqueName: \"kubernetes.io/projected/7d49eb22-75ca-4458-95ef-c54eeaae9ff3-kube-api-access-zlh8j\") pod \"test-curl-7928-140115462225040\" (UID: \"7d49eb22-75ca-4458-95ef-c54eeaae9ff3\") " pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:03.991538 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:03.991496    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlh8j\" (UniqueName: \"kubernetes.io/projected/7d49eb22-75ca-4458-95ef-c54eeaae9ff3-kube-api-access-zlh8j\") pod \"test-curl-7928-140115462225040\" (UID: \"7d49eb22-75ca-4458-95ef-c54eeaae9ff3\") " pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:04.000984 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.000953    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlh8j\" (UniqueName: \"kubernetes.io/projected/7d49eb22-75ca-4458-95ef-c54eeaae9ff3-kube-api-access-zlh8j\") pod \"test-curl-7928-140115462225040\" (UID: \"7d49eb22-75ca-4458-95ef-c54eeaae9ff3\") " pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:04.100655 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.100561    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:04.237048 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.237020    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:04.238666 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:17:04.238641    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d49eb22_75ca_4458_95ef_c54eeaae9ff3.slice/crio-db4516bbe758c1f05ff73d6b435ae5a93dccdd41999a0523945e156bceebd2ef WatchSource:0}: Error finding container db4516bbe758c1f05ff73d6b435ae5a93dccdd41999a0523945e156bceebd2ef: Status 404 returned error can't find the container with id db4516bbe758c1f05ff73d6b435ae5a93dccdd41999a0523945e156bceebd2ef
Jul 29 22:17:04.460697 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.460664    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462225040" event={"ID":"7d49eb22-75ca-4458-95ef-c54eeaae9ff3","Type":"ContainerStarted","Data":"db4516bbe758c1f05ff73d6b435ae5a93dccdd41999a0523945e156bceebd2ef"}
Jul 29 22:17:04.461920 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.461895    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:04.462056 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.461898    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="287153e2a99a9122dac50c8bee4244281cdd67ffb3c5c1259461a8d49154e0b7"
Jul 29 22:17:04.476423 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:04.476387    2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7928-140115462225040" oldPodUID="d632e1cb-3b95-4e68-a64a-e6347c183782" podUID="7d49eb22-75ca-4458-95ef-c54eeaae9ff3"
Jul 29 22:17:05.304750 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:05.304720    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d632e1cb-3b95-4e68-a64a-e6347c183782" path="/var/lib/kubelet/pods/d632e1cb-3b95-4e68-a64a-e6347c183782/volumes"
Jul 29 22:17:05.471510 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:05.471476    2572 generic.go:356] "Generic (PLEG): container finished" podID="7d49eb22-75ca-4458-95ef-c54eeaae9ff3" containerID="7ba9e4eb3b08d3d117d874ae13e17544640e4b241526fc87ab919e64faa2de39" exitCode=0
Jul 29 22:17:05.471703 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:05.471564    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462225040" event={"ID":"7d49eb22-75ca-4458-95ef-c54eeaae9ff3","Type":"ContainerDied","Data":"7ba9e4eb3b08d3d117d874ae13e17544640e4b241526fc87ab919e64faa2de39"}
Jul 29 22:17:06.626146 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.626120    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:06.648332 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.648303    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7928-140115462225040_7d49eb22-75ca-4458-95ef-c54eeaae9ff3/test-curl-7928-140115462225040/0.log"
Jul 29 22:17:06.671121 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.671079    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:06.674892 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.674864    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7928-140115462225040"]
Jul 29 22:17:06.715663 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.715630    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlh8j\" (UniqueName: \"kubernetes.io/projected/7d49eb22-75ca-4458-95ef-c54eeaae9ff3-kube-api-access-zlh8j\") pod \"7d49eb22-75ca-4458-95ef-c54eeaae9ff3\" (UID: \"7d49eb22-75ca-4458-95ef-c54eeaae9ff3\") "
Jul 29 22:17:06.718068 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.718028    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d49eb22-75ca-4458-95ef-c54eeaae9ff3-kube-api-access-zlh8j" (OuterVolumeSpecName: "kube-api-access-zlh8j") pod "7d49eb22-75ca-4458-95ef-c54eeaae9ff3" (UID: "7d49eb22-75ca-4458-95ef-c54eeaae9ff3"). InnerVolumeSpecName "kube-api-access-zlh8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:17:06.801901 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.801818    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:06.802305 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.802291    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d49eb22-75ca-4458-95ef-c54eeaae9ff3" containerName="test-curl-7928-140115462225040"
Jul 29 22:17:06.802364 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.802306    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d49eb22-75ca-4458-95ef-c54eeaae9ff3" containerName="test-curl-7928-140115462225040"
Jul 29 22:17:06.802402 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.802374    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d49eb22-75ca-4458-95ef-c54eeaae9ff3" containerName="test-curl-7928-140115462225040"
Jul 29 22:17:06.805283 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.805259    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:06.811718 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.811686    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:06.816496 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.816470    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlh8j\" (UniqueName: \"kubernetes.io/projected/7d49eb22-75ca-4458-95ef-c54eeaae9ff3-kube-api-access-zlh8j\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:06.917920 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:06.917876    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srdrr\" (UniqueName: \"kubernetes.io/projected/ef714faa-a0c5-4a1b-8182-3679ebd7cb00-kube-api-access-srdrr\") pod \"test-curl-7928-140115462230064\" (UID: \"ef714faa-a0c5-4a1b-8182-3679ebd7cb00\") " pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:07.018909 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.018868    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srdrr\" (UniqueName: \"kubernetes.io/projected/ef714faa-a0c5-4a1b-8182-3679ebd7cb00-kube-api-access-srdrr\") pod \"test-curl-7928-140115462230064\" (UID: \"ef714faa-a0c5-4a1b-8182-3679ebd7cb00\") " pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:07.027843 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.027815    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srdrr\" (UniqueName: \"kubernetes.io/projected/ef714faa-a0c5-4a1b-8182-3679ebd7cb00-kube-api-access-srdrr\") pod \"test-curl-7928-140115462230064\" (UID: \"ef714faa-a0c5-4a1b-8182-3679ebd7cb00\") " pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:07.116236 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.116113    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:07.261477 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.261443    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:07.261858 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:17:07.261835    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef714faa_a0c5_4a1b_8182_3679ebd7cb00.slice/crio-a55b1b1a23e4cf23c7b3ebfbe5030b4fd53702728d83a02ed23fcffc0460f4f1 WatchSource:0}: Error finding container a55b1b1a23e4cf23c7b3ebfbe5030b4fd53702728d83a02ed23fcffc0460f4f1: Status 404 returned error can't find the container with id a55b1b1a23e4cf23c7b3ebfbe5030b4fd53702728d83a02ed23fcffc0460f4f1
Jul 29 22:17:07.304687 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.304654    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d49eb22-75ca-4458-95ef-c54eeaae9ff3" path="/var/lib/kubelet/pods/7d49eb22-75ca-4458-95ef-c54eeaae9ff3/volumes"
Jul 29 22:17:07.485007 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.484968    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462230064" event={"ID":"ef714faa-a0c5-4a1b-8182-3679ebd7cb00","Type":"ContainerStarted","Data":"a55b1b1a23e4cf23c7b3ebfbe5030b4fd53702728d83a02ed23fcffc0460f4f1"}
Jul 29 22:17:07.486447 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.486422    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462225040"
Jul 29 22:17:07.486447 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:07.486406    2572 scope.go:117] "RemoveContainer" containerID="7ba9e4eb3b08d3d117d874ae13e17544640e4b241526fc87ab919e64faa2de39"
Jul 29 22:17:08.494618 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:08.494586    2572 generic.go:356] "Generic (PLEG): container finished" podID="ef714faa-a0c5-4a1b-8182-3679ebd7cb00" containerID="a3b78e087b6ea818fcfb6e370b7e8f0c12a20748f079a0c60914eeee2d61a5c1" exitCode=0
Jul 29 22:17:08.495006 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:08.494638    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462230064" event={"ID":"ef714faa-a0c5-4a1b-8182-3679ebd7cb00","Type":"ContainerDied","Data":"a3b78e087b6ea818fcfb6e370b7e8f0c12a20748f079a0c60914eeee2d61a5c1"}
Jul 29 22:17:08.819903 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:08.819808    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn" podUID="92e36b02-8eba-4dba-b004-36f2f443e8e4" containerName="main" containerID="cri-o://472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8" gracePeriod=45
Jul 29 22:17:09.106019 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.105995    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:17:09.239504 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239466    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-tmp-dir\") pod \"92e36b02-8eba-4dba-b004-36f2f443e8e4\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") "
Jul 29 22:17:09.239700 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239568    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e36b02-8eba-4dba-b004-36f2f443e8e4-tls-certs\") pod \"92e36b02-8eba-4dba-b004-36f2f443e8e4\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") "
Jul 29 22:17:09.239700 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239617    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2kdp\" (UniqueName: \"kubernetes.io/projected/92e36b02-8eba-4dba-b004-36f2f443e8e4-kube-api-access-g2kdp\") pod \"92e36b02-8eba-4dba-b004-36f2f443e8e4\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") "
Jul 29 22:17:09.239700 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239653    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-home\") pod \"92e36b02-8eba-4dba-b004-36f2f443e8e4\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") "
Jul 29 22:17:09.239876 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239702    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-dshm\") pod \"92e36b02-8eba-4dba-b004-36f2f443e8e4\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") "
Jul 29 22:17:09.239876 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239742    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-model-cache\") pod \"92e36b02-8eba-4dba-b004-36f2f443e8e4\" (UID: \"92e36b02-8eba-4dba-b004-36f2f443e8e4\") "
Jul 29 22:17:09.239876 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.239779    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "92e36b02-8eba-4dba-b004-36f2f443e8e4" (UID: "92e36b02-8eba-4dba-b004-36f2f443e8e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:17:09.240042 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.240008    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-home" (OuterVolumeSpecName: "home") pod "92e36b02-8eba-4dba-b004-36f2f443e8e4" (UID: "92e36b02-8eba-4dba-b004-36f2f443e8e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:17:09.240042 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.240036    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.240148 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.240041    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "92e36b02-8eba-4dba-b004-36f2f443e8e4" (UID: "92e36b02-8eba-4dba-b004-36f2f443e8e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:17:09.241836 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.241804    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-dshm" (OuterVolumeSpecName: "dshm") pod "92e36b02-8eba-4dba-b004-36f2f443e8e4" (UID: "92e36b02-8eba-4dba-b004-36f2f443e8e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:17:09.241941 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.241852    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92e36b02-8eba-4dba-b004-36f2f443e8e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "92e36b02-8eba-4dba-b004-36f2f443e8e4" (UID: "92e36b02-8eba-4dba-b004-36f2f443e8e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:17:09.241941 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.241873    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92e36b02-8eba-4dba-b004-36f2f443e8e4-kube-api-access-g2kdp" (OuterVolumeSpecName: "kube-api-access-g2kdp") pod "92e36b02-8eba-4dba-b004-36f2f443e8e4" (UID: "92e36b02-8eba-4dba-b004-36f2f443e8e4"). InnerVolumeSpecName "kube-api-access-g2kdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:17:09.341553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.341455    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92e36b02-8eba-4dba-b004-36f2f443e8e4-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.341553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.341488    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2kdp\" (UniqueName: \"kubernetes.io/projected/92e36b02-8eba-4dba-b004-36f2f443e8e4-kube-api-access-g2kdp\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.341553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.341501    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.341553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.341513    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.341553 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.341523    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/92e36b02-8eba-4dba-b004-36f2f443e8e4-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.501390 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.501353    2572 generic.go:356] "Generic (PLEG): container finished" podID="92e36b02-8eba-4dba-b004-36f2f443e8e4" containerID="472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8" exitCode=0
Jul 29 22:17:09.501390 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.501393    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"
Jul 29 22:17:09.501929 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.501435    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn" event={"ID":"92e36b02-8eba-4dba-b004-36f2f443e8e4","Type":"ContainerDied","Data":"472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8"}
Jul 29 22:17:09.501929 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.501478    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn" event={"ID":"92e36b02-8eba-4dba-b004-36f2f443e8e4","Type":"ContainerDied","Data":"f50a7e4e9d29264ea8de0751cdad68a5428c7d48c4ea02b72a1bfced0a2745ca"}
Jul 29 22:17:09.501929 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.501498    2572 scope.go:117] "RemoveContainer" containerID="472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8"
Jul 29 22:17:09.511005 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.510980    2572 scope.go:117] "RemoveContainer" containerID="472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8"
Jul 29 22:17:09.511322 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:17:09.511297    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8\": container with ID starting with 472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8 not found: ID does not exist" containerID="472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8"
Jul 29 22:17:09.511402 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.511334    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8"} err="failed to get container status \"472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8\": rpc error: code = NotFound desc = could not find container \"472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8\": container with ID starting with 472b5e333bf33b3d996cdae67842437aa4d5736783557050b8d424236a0c11a8 not found: ID does not exist"
Jul 29 22:17:09.523056 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.523028    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"]
Jul 29 22:17:09.526879 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.526854    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-ca895e/rate-test-model-ca895e-kserve-779f9fb9f9-xcqcn"]
Jul 29 22:17:09.632032 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.632009    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:09.651808 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.651784    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7928-140115462230064_ef714faa-a0c5-4a1b-8182-3679ebd7cb00/test-curl-7928-140115462230064/0.log"
Jul 29 22:17:09.676511 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.676484    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:09.680940 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.680910    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:09.744493 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.744459    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srdrr\" (UniqueName: \"kubernetes.io/projected/ef714faa-a0c5-4a1b-8182-3679ebd7cb00-kube-api-access-srdrr\") pod \"ef714faa-a0c5-4a1b-8182-3679ebd7cb00\" (UID: \"ef714faa-a0c5-4a1b-8182-3679ebd7cb00\") "
Jul 29 22:17:09.746542 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.746519    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef714faa-a0c5-4a1b-8182-3679ebd7cb00-kube-api-access-srdrr" (OuterVolumeSpecName: "kube-api-access-srdrr") pod "ef714faa-a0c5-4a1b-8182-3679ebd7cb00" (UID: "ef714faa-a0c5-4a1b-8182-3679ebd7cb00"). InnerVolumeSpecName "kube-api-access-srdrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:17:09.805164 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805127    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:09.805547 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805534    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef714faa-a0c5-4a1b-8182-3679ebd7cb00" containerName="test-curl-7928-140115462230064"
Jul 29 22:17:09.805589 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805549    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef714faa-a0c5-4a1b-8182-3679ebd7cb00" containerName="test-curl-7928-140115462230064"
Jul 29 22:17:09.805589 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805565    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92e36b02-8eba-4dba-b004-36f2f443e8e4" containerName="main"
Jul 29 22:17:09.805589 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805570    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="92e36b02-8eba-4dba-b004-36f2f443e8e4" containerName="main"
Jul 29 22:17:09.805681 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805642    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef714faa-a0c5-4a1b-8182-3679ebd7cb00" containerName="test-curl-7928-140115462230064"
Jul 29 22:17:09.805681 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.805655    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="92e36b02-8eba-4dba-b004-36f2f443e8e4" containerName="main"
Jul 29 22:17:09.808716 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.808700    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:09.814634 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.814610    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:09.845693 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.845662    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srdrr\" (UniqueName: \"kubernetes.io/projected/ef714faa-a0c5-4a1b-8182-3679ebd7cb00-kube-api-access-srdrr\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:09.946519 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:09.946484    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m48jq\" (UniqueName: \"kubernetes.io/projected/27c6c454-4b7c-444b-927f-767d5bca356f-kube-api-access-m48jq\") pod \"test-curl-7928-140115462230064\" (UID: \"27c6c454-4b7c-444b-927f-767d5bca356f\") " pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:10.047854 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.047812    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m48jq\" (UniqueName: \"kubernetes.io/projected/27c6c454-4b7c-444b-927f-767d5bca356f-kube-api-access-m48jq\") pod \"test-curl-7928-140115462230064\" (UID: \"27c6c454-4b7c-444b-927f-767d5bca356f\") " pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:10.056754 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.056719    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m48jq\" (UniqueName: \"kubernetes.io/projected/27c6c454-4b7c-444b-927f-767d5bca356f-kube-api-access-m48jq\") pod \"test-curl-7928-140115462230064\" (UID: \"27c6c454-4b7c-444b-927f-767d5bca356f\") " pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:10.120038 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.119996    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:10.256117 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.256089    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:10.257999 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:17:10.257967    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27c6c454_4b7c_444b_927f_767d5bca356f.slice/crio-5043d987d076f44548c67dce5855dae15f73863cb07dfb4f30593c53e6a96aa9 WatchSource:0}: Error finding container 5043d987d076f44548c67dce5855dae15f73863cb07dfb4f30593c53e6a96aa9: Status 404 returned error can't find the container with id 5043d987d076f44548c67dce5855dae15f73863cb07dfb4f30593c53e6a96aa9
Jul 29 22:17:10.508541 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.508449    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:10.508938 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.508459    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a55b1b1a23e4cf23c7b3ebfbe5030b4fd53702728d83a02ed23fcffc0460f4f1"
Jul 29 22:17:10.510914 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.510886    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462230064" event={"ID":"27c6c454-4b7c-444b-927f-767d5bca356f","Type":"ContainerStarted","Data":"5043d987d076f44548c67dce5855dae15f73863cb07dfb4f30593c53e6a96aa9"}
Jul 29 22:17:10.520578 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:10.520547    2572 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7928-140115462230064" oldPodUID="ef714faa-a0c5-4a1b-8182-3679ebd7cb00" podUID="27c6c454-4b7c-444b-927f-767d5bca356f"
Jul 29 22:17:11.304702 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:11.304653    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92e36b02-8eba-4dba-b004-36f2f443e8e4" path="/var/lib/kubelet/pods/92e36b02-8eba-4dba-b004-36f2f443e8e4/volumes"
Jul 29 22:17:11.305111 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:11.305090    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef714faa-a0c5-4a1b-8182-3679ebd7cb00" path="/var/lib/kubelet/pods/ef714faa-a0c5-4a1b-8182-3679ebd7cb00/volumes"
Jul 29 22:17:11.517961 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:11.517922    2572 generic.go:356] "Generic (PLEG): container finished" podID="27c6c454-4b7c-444b-927f-767d5bca356f" containerID="de4c2807f81510ce8033ed1edfa711de8a833de65ff108f81044d85c46a0d623" exitCode=0
Jul 29 22:17:11.518422 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:11.518016    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7928-140115462230064" event={"ID":"27c6c454-4b7c-444b-927f-767d5bca356f","Type":"ContainerDied","Data":"de4c2807f81510ce8033ed1edfa711de8a833de65ff108f81044d85c46a0d623"}
Jul 29 22:17:12.660367 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.660342    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:12.679736 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.679706    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7928-140115462230064_27c6c454-4b7c-444b-927f-767d5bca356f/test-curl-7928-140115462230064/0.log"
Jul 29 22:17:12.705751 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.705715    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:12.711332 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.711310    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7928-140115462230064"]
Jul 29 22:17:12.770813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.770785    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m48jq\" (UniqueName: \"kubernetes.io/projected/27c6c454-4b7c-444b-927f-767d5bca356f-kube-api-access-m48jq\") pod \"27c6c454-4b7c-444b-927f-767d5bca356f\" (UID: \"27c6c454-4b7c-444b-927f-767d5bca356f\") "
Jul 29 22:17:12.772744 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.772721    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c6c454-4b7c-444b-927f-767d5bca356f-kube-api-access-m48jq" (OuterVolumeSpecName: "kube-api-access-m48jq") pod "27c6c454-4b7c-444b-927f-767d5bca356f" (UID: "27c6c454-4b7c-444b-927f-767d5bca356f"). InnerVolumeSpecName "kube-api-access-m48jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:17:12.872360 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:12.872330    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m48jq\" (UniqueName: \"kubernetes.io/projected/27c6c454-4b7c-444b-927f-767d5bca356f-kube-api-access-m48jq\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:17:13.238263 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.238227    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:13.304979 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.304946    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c6c454-4b7c-444b-927f-767d5bca356f" path="/var/lib/kubelet/pods/27c6c454-4b7c-444b-927f-767d5bca356f/volumes"
Jul 29 22:17:13.534892 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.534696    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7928-140115462230064"
Jul 29 22:17:13.534892 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.534740    2572 scope.go:117] "RemoveContainer" containerID="de4c2807f81510ce8033ed1edfa711de8a833de65ff108f81044d85c46a0d623"
Jul 29 22:17:13.536114 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.536091    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"]
Jul 29 22:17:13.536964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.536939    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c6c454-4b7c-444b-927f-767d5bca356f" containerName="test-curl-7928-140115462230064"
Jul 29 22:17:13.536964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.536964    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c6c454-4b7c-444b-927f-767d5bca356f" containerName="test-curl-7928-140115462230064"
Jul 29 22:17:13.537138 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.537078    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c6c454-4b7c-444b-927f-767d5bca356f" containerName="test-curl-7928-140115462230064"
Jul 29 22:17:13.540740 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.540717    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.543244 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.543218    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 22:17:13.543351 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.543273    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-ee9d15-openshift-default-dockercfg-8fg5w\""
Jul 29 22:17:13.552907 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.552651    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"]
Jul 29 22:17:13.565872 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.565842    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:13.681371 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-envoy\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681371 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681377    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-socket\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681423    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-podinfo\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681439    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-token\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681498    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istiod-ca-cert\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681523    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l65lx\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-kube-api-access-l65lx\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681548    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-data\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681573    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-certs\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.681855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.681598    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-credential-socket\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.723194 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.723124    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:13.782637 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782604    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-envoy\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782637 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782639    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-socket\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782671    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-podinfo\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782714    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-token\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782751    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istiod-ca-cert\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782773    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l65lx\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-kube-api-access-l65lx\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782803    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-data\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.782844 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782823    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-certs\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.783123 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.782844    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-credential-socket\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.783334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.783273    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-socket\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.783334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.783329    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-credential-socket\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.783538 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.783386    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-certs\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.783538 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.783454    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-data\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.783538 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.783524    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istiod-ca-cert\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.785195 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.785129    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-envoy\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.785422 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.785403    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-podinfo\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.790323 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.790304    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-token\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.790578 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.790562    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l65lx\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-kube-api-access-l65lx\") pod \"e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") " pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:13.856960 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:13.856917    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:14.010950 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:14.010919    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"]
Jul 29 22:17:14.012708 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:17:14.012682    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf2abd8_fc2a_4053_b420_b1cc0a96e0b7.slice/crio-2b0b4057f26e0471ca65fbd590565c14440f8d4bc59cb732862428a493d0d06a WatchSource:0}: Error finding container 2b0b4057f26e0471ca65fbd590565c14440f8d4bc59cb732862428a493d0d06a: Status 404 returned error can't find the container with id 2b0b4057f26e0471ca65fbd590565c14440f8d4bc59cb732862428a493d0d06a
Jul 29 22:17:14.542257 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:14.542215    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" event={"ID":"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7","Type":"ContainerStarted","Data":"2b0b4057f26e0471ca65fbd590565c14440f8d4bc59cb732862428a493d0d06a"}
Jul 29 22:17:17.252338 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:17.252303    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 22:17:17.252599 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:17.252376    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 22:17:17.252599 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:17.252404    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 22:17:17.567900 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:17.567797    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" event={"ID":"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7","Type":"ContainerStarted","Data":"a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"}
Jul 29 22:17:17.592652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:17.592598    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" podStartSLOduration=1.355330354 podStartE2EDuration="4.592581903s" totalImagesPullingTime="3.237251549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:17:17.589958096 +0000 UTC m=+3648.806735559" watchObservedRunningTime="2026-07-29 22:17:17.592581903 +0000 UTC m=+3648.809359365"
Jul 29 22:17:17.857965 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:17.857856    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:18.862904 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:18.862875    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:19.545843 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:19.545794    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:19.580071 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:19.580043    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:19.581232 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:19.581210    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:19.646766 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:19.646733    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:20.722008 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:20.721955    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:20.924002 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:20.923965    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:21.332651 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:21.332613    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:21.433678 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:21.433639    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:24.332066 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:24.332024    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:17:32.658729 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:32.658632    2572 generic.go:356] "Generic (PLEG): container finished" podID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerID="a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484" exitCode=0
Jul 29 22:17:32.658729 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:32.658694    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" event={"ID":"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7","Type":"ContainerDied","Data":"a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"}
Jul 29 22:17:32.659164 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:32.659049    2572 scope.go:117] "RemoveContainer" containerID="a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"
Jul 29 22:17:32.660144 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:32.660116    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 22:17:32.660223 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:32.660208    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 22:17:32.660268 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:32.660244    2572 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 29 22:17:33.669430 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:33.669379    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" event={"ID":"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7","Type":"ContainerStarted","Data":"379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0"}
Jul 29 22:17:33.857926 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:33.857888    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:33.859130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:33.859103    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Jul 29 22:17:33.859275 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:33.859150    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Jul 29 22:17:34.580152 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:34.580111    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:34.857608 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:34.857514    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Jul 29 22:17:34.857608 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:34.857577    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Jul 29 22:17:35.858066 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:35.858023    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Jul 29 22:17:35.858505 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:35.858093    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Jul 29 22:17:36.858162 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:36.858123    2572 patch_prober.go:28] interesting pod/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Jul 29 22:17:36.858587 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:36.858232    2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Jul 29 22:17:37.862117 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:37.862076    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:37.863114 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:37.863092    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:17:38.685505 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:38.685471    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:39.896834 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:39.896791    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:44.568019 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:44.567985    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:45.306645 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:45.306611    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:45.636231 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:45.636119    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:46.149574 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:46.149541    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:48.820862 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:48.820826    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:48.903511 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:48.903465    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:49.944164 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:49.944129    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:50.078839 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:50.078801    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:50.551393 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:50.551359    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:50.652752 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:50.652714    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:17:58.139367 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:17:58.139275    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:18:13.883935 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:13.883901    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:18:29.205627 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.205594    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"]
Jul 29 22:18:29.209422 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.209396    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.212785 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.212763    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ee9d15\"/\"ipp-route-ee9d15-kserve-self-signed-certs\""
Jul 29 22:18:29.212924 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.212770    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ee9d15\"/\"openshift-service-ca.crt\""
Jul 29 22:18:29.212924 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.212770    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-ee9d15\"/\"default-dockercfg-59ck9\""
Jul 29 22:18:29.213133 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.213116    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-ee9d15\"/\"kube-root-ca.crt\""
Jul 29 22:18:29.220537 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.220519    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"]
Jul 29 22:18:29.306751 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.306716    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:18:29.321932 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.321908    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tls-certs\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.322018 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.321953    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-home\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.322057 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.322015    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tmp-dir\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.322057 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.322047    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf6fq\" (UniqueName: \"kubernetes.io/projected/a2415aa7-2d70-435a-9f4a-f395d39d3f07-kube-api-access-hf6fq\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.322129 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.322068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-dshm\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.322129 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.322114    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-model-cache\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423477 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423438    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-model-cache\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423650 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423626    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tls-certs\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423730 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423714    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-home\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423781 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423764    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tmp-dir\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423830 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423801    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf6fq\" (UniqueName: \"kubernetes.io/projected/a2415aa7-2d70-435a-9f4a-f395d39d3f07-kube-api-access-hf6fq\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423880 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423831    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-dshm\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.423938 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.423924    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-model-cache\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.424063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.424041    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-home\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.424144 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.424123    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tmp-dir\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.425914 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.425894    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-dshm\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.426103 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.426084    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tls-certs\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.442034 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.442012    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf6fq\" (UniqueName: \"kubernetes.io/projected/a2415aa7-2d70-435a-9f4a-f395d39d3f07-kube-api-access-hf6fq\") pod \"ipp-route-ee9d15-kserve-6875777887-94s5j\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") " pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.519874 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.519792    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:29.661122 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:29.661089    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"]
Jul 29 22:18:29.663369 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:18:29.663339    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2415aa7_2d70_435a_9f4a_f395d39d3f07.slice/crio-8ee01fb267acf7b239288836bf4b434463fa8b80bc4f381f448e5acd7272ca4b WatchSource:0}: Error finding container 8ee01fb267acf7b239288836bf4b434463fa8b80bc4f381f448e5acd7272ca4b: Status 404 returned error can't find the container with id 8ee01fb267acf7b239288836bf4b434463fa8b80bc4f381f448e5acd7272ca4b
Jul 29 22:18:30.012366 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:30.012331    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j" event={"ID":"a2415aa7-2d70-435a-9f4a-f395d39d3f07","Type":"ContainerStarted","Data":"21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9"}
Jul 29 22:18:30.012366 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:30.012372    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j" event={"ID":"a2415aa7-2d70-435a-9f4a-f395d39d3f07","Type":"ContainerStarted","Data":"8ee01fb267acf7b239288836bf4b434463fa8b80bc4f381f448e5acd7272ca4b"}
Jul 29 22:18:30.030832 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:30.030782    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j" podStartSLOduration=1.030767467 podStartE2EDuration="1.030767467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:18:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:18:30.028471498 +0000 UTC m=+3721.245248960" watchObservedRunningTime="2026-07-29 22:18:30.030767467 +0000 UTC m=+3721.247544928"
Jul 29 22:18:33.328245 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:33.328202    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:18:39.520646 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:39.520601    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:39.521293 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:39.520671    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:39.533887 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:39.533859    2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:18:40.087532 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:18:40.087495    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:19:06.321311 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:06.321263    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:19:16.772329 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:16.772285    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:17.174039 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:17.173954    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:17.411994 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:17.411959    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:20.342855 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:20.342815    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:20.443194 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:20.443142    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:21.440066 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.440027    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"]
Jul 29 22:19:21.444297 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.444275    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.446842 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.446819    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:19:21.446969 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.446863    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 22:19:21.447530 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.447510    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-eafc3c-serving-cert\""
Jul 29 22:19:21.447603 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.447534    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-p4qdf\""
Jul 29 22:19:21.455350 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.455324    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"]
Jul 29 22:19:21.468291 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.468266    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:21.527424 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.527335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.527606 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.527457    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqf7f\" (UniqueName: \"kubernetes.io/projected/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-kube-api-access-pqf7f\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.527606 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.527522    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.628114 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.628075    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.628312 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.628212    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqf7f\" (UniqueName: \"kubernetes.io/projected/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-kube-api-access-pqf7f\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.628312 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.628245    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.628404 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:19:21.628364    2572 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-serving-cert: secret "maas-api-e2e-ipp-cleanup-eafc3c-serving-cert" not found
Jul 29 22:19:21.628453 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:19:21.628443    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls podName:c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e nodeName:}" failed. No retries permitted until 2026-07-29 22:19:22.128425423 +0000 UTC m=+3773.345202867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" (UID: "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e") : secret "maas-api-e2e-ipp-cleanup-eafc3c-serving-cert" not found
Jul 29 22:19:21.628732 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.628712    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.637307 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.637285    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqf7f\" (UniqueName: \"kubernetes.io/projected/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-kube-api-access-pqf7f\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:21.733838 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:21.733749    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:22.073925 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.073834    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:22.134347 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.134303    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:22.136760 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.136743    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:22.169063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.169024    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:22.211585 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.211548    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"]
Jul 29 22:19:22.219269 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.219008    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.221667 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.221640    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-eafc3c-dockercfg-rjph5\""
Jul 29 22:19:22.221846 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.221723    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-eafc3c\""
Jul 29 22:19:22.223246 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.223221    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"]
Jul 29 22:19:22.336694 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.336609    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/282cac88-fac8-4361-a501-732e00725b8b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.336694 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.336677    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrn2n\" (UniqueName: \"kubernetes.io/projected/282cac88-fac8-4361-a501-732e00725b8b-kube-api-access-zrn2n\") pod \"payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.358764 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.358730    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:22.437634 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.437545    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/282cac88-fac8-4361-a501-732e00725b8b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.437634 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.437603    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrn2n\" (UniqueName: \"kubernetes.io/projected/282cac88-fac8-4361-a501-732e00725b8b-kube-api-access-zrn2n\") pod \"payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.438438 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.438409    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/282cac88-fac8-4361-a501-732e00725b8b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.446799 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.446773    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrn2n\" (UniqueName: \"kubernetes.io/projected/282cac88-fac8-4361-a501-732e00725b8b-kube-api-access-zrn2n\") pod \"payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.510757 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.510725    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"]
Jul 29 22:19:22.512377 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:19:22.512343    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ea4e6e_e93a_4502_b5d6_dcbb9df59f1e.slice/crio-c448552a2aa187faa9563fbf25abe22409a85c35ce35edb45ba5b4488fcffb53 WatchSource:0}: Error finding container c448552a2aa187faa9563fbf25abe22409a85c35ce35edb45ba5b4488fcffb53: Status 404 returned error can't find the container with id c448552a2aa187faa9563fbf25abe22409a85c35ce35edb45ba5b4488fcffb53
Jul 29 22:19:22.533017 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.532985    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:22.675462 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:22.675435    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"]
Jul 29 22:19:22.677538 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:19:22.677510    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod282cac88_fac8_4361_a501_732e00725b8b.slice/crio-53fdbf44f1d371da42a61573b4558e17856a15e989510064f7a6ad8728c2cbd3 WatchSource:0}: Error finding container 53fdbf44f1d371da42a61573b4558e17856a15e989510064f7a6ad8728c2cbd3: Status 404 returned error can't find the container with id 53fdbf44f1d371da42a61573b4558e17856a15e989510064f7a6ad8728c2cbd3
Jul 29 22:19:23.348679 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:23.348118    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk" event={"ID":"282cac88-fac8-4361-a501-732e00725b8b","Type":"ContainerStarted","Data":"76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032"}
Jul 29 22:19:23.348679 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:23.348206    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk" event={"ID":"282cac88-fac8-4361-a501-732e00725b8b","Type":"ContainerStarted","Data":"53fdbf44f1d371da42a61573b4558e17856a15e989510064f7a6ad8728c2cbd3"}
Jul 29 22:19:23.348679 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:23.348347    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:23.350131 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:23.350100    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" event={"ID":"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e","Type":"ContainerStarted","Data":"c448552a2aa187faa9563fbf25abe22409a85c35ce35edb45ba5b4488fcffb53"}
Jul 29 22:19:23.367604 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:23.366393    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk" podStartSLOduration=1.366368969 podStartE2EDuration="1.366368969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:19:23.363129568 +0000 UTC m=+3774.579907031" watchObservedRunningTime="2026-07-29 22:19:23.366368969 +0000 UTC m=+3774.583146434"
Jul 29 22:19:25.366265 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:25.366151    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" event={"ID":"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e","Type":"ContainerStarted","Data":"f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6"}
Jul 29 22:19:25.366265 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:25.366242    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:25.382687 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:25.382593    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" podStartSLOduration=1.833639496 podStartE2EDuration="4.382576711s" totalImagesPullingTime="2.548937215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:19:25.382139692 +0000 UTC m=+3776.598917155" watchObservedRunningTime="2026-07-29 22:19:25.382576711 +0000 UTC m=+3776.599354174"
Jul 29 22:19:25.819416 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:25.819379    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:19:31.381074 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:31.381038    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:34.359865 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:34.359830    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:45.428605 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:45.428564    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:51.900354 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:51.900311    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"]
Jul 29 22:19:51.900964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:51.900638    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" podUID="c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" containerName="maas-api" containerID="cri-o://f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6" gracePeriod=30
Jul 29 22:19:51.939915 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:51.939877    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:51.963484 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:51.963449    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"]
Jul 29 22:19:51.963737 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:51.963667    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk" podUID="282cac88-fac8-4361-a501-732e00725b8b" containerName="payload-processing" containerID="cri-o://76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032" gracePeriod=30
Jul 29 22:19:52.040481 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.040440    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:52.111989 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.111957    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:52.218888 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.218853    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:52.252762 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.252729    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:52.338980 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.338947    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls\") pod \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") "
Jul 29 22:19:52.339228 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.338998    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqf7f\" (UniqueName: \"kubernetes.io/projected/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-kube-api-access-pqf7f\") pod \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") "
Jul 29 22:19:52.339228 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.339032    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrn2n\" (UniqueName: \"kubernetes.io/projected/282cac88-fac8-4361-a501-732e00725b8b-kube-api-access-zrn2n\") pod \"282cac88-fac8-4361-a501-732e00725b8b\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") "
Jul 29 22:19:52.339228 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.339122    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-openshift-service-ca\") pod \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\" (UID: \"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e\") "
Jul 29 22:19:52.339228 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.339152    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/282cac88-fac8-4361-a501-732e00725b8b-plugins-config-volume\") pod \"282cac88-fac8-4361-a501-732e00725b8b\" (UID: \"282cac88-fac8-4361-a501-732e00725b8b\") "
Jul 29 22:19:52.339636 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.339590    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" (UID: "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:19:52.339768 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.339692    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/282cac88-fac8-4361-a501-732e00725b8b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "282cac88-fac8-4361-a501-732e00725b8b" (UID: "282cac88-fac8-4361-a501-732e00725b8b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:19:52.341218 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.341195    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/282cac88-fac8-4361-a501-732e00725b8b-kube-api-access-zrn2n" (OuterVolumeSpecName: "kube-api-access-zrn2n") pod "282cac88-fac8-4361-a501-732e00725b8b" (UID: "282cac88-fac8-4361-a501-732e00725b8b"). InnerVolumeSpecName "kube-api-access-zrn2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:19:52.341353 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.341326    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-kube-api-access-pqf7f" (OuterVolumeSpecName: "kube-api-access-pqf7f") pod "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" (UID: "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e"). InnerVolumeSpecName "kube-api-access-pqf7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:19:52.341409 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.341347    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" (UID: "c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:19:52.440795 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.440751    2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-openshift-service-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:19:52.440795 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.440792    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/282cac88-fac8-4361-a501-732e00725b8b-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:19:52.441063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.440806    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-maas-api-tls\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:19:52.441063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.440821    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqf7f\" (UniqueName: \"kubernetes.io/projected/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e-kube-api-access-pqf7f\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:19:52.441063 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.440835    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrn2n\" (UniqueName: \"kubernetes.io/projected/282cac88-fac8-4361-a501-732e00725b8b-kube-api-access-zrn2n\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:19:52.504805 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.504767    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:52.530580 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.530549    2572 generic.go:356] "Generic (PLEG): container finished" podID="282cac88-fac8-4361-a501-732e00725b8b" containerID="76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032" exitCode=0
Jul 29 22:19:52.530759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.530601    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"
Jul 29 22:19:52.530759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.530633    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk" event={"ID":"282cac88-fac8-4361-a501-732e00725b8b","Type":"ContainerDied","Data":"76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032"}
Jul 29 22:19:52.530759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.530683    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk" event={"ID":"282cac88-fac8-4361-a501-732e00725b8b","Type":"ContainerDied","Data":"53fdbf44f1d371da42a61573b4558e17856a15e989510064f7a6ad8728c2cbd3"}
Jul 29 22:19:52.530759 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.530712    2572 scope.go:117] "RemoveContainer" containerID="76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032"
Jul 29 22:19:52.532150 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.532127    2572 generic.go:356] "Generic (PLEG): container finished" podID="c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" containerID="f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6" exitCode=0
Jul 29 22:19:52.532281 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.532154    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" event={"ID":"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e","Type":"ContainerDied","Data":"f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6"}
Jul 29 22:19:52.532281 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.532195    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"
Jul 29 22:19:52.532281 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.532225    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f" event={"ID":"c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e","Type":"ContainerDied","Data":"c448552a2aa187faa9563fbf25abe22409a85c35ce35edb45ba5b4488fcffb53"}
Jul 29 22:19:52.540248 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.540225    2572 scope.go:117] "RemoveContainer" containerID="76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032"
Jul 29 22:19:52.540568 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:19:52.540542    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032\": container with ID starting with 76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032 not found: ID does not exist" containerID="76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032"
Jul 29 22:19:52.540682 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.540573    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032"} err="failed to get container status \"76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032\": rpc error: code = NotFound desc = could not find container \"76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032\": container with ID starting with 76441d55a3b91f0fae2085b187a1acd702782061e4730c3d2c8c1dff45eda032 not found: ID does not exist"
Jul 29 22:19:52.540682 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.540591    2572 scope.go:117] "RemoveContainer" containerID="f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6"
Jul 29 22:19:52.549322 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.549304    2572 scope.go:117] "RemoveContainer" containerID="f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6"
Jul 29 22:19:52.549569 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:19:52.549550    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6\": container with ID starting with f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6 not found: ID does not exist" containerID="f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6"
Jul 29 22:19:52.549644 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.549574    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6"} err="failed to get container status \"f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6\": rpc error: code = NotFound desc = could not find container \"f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6\": container with ID starting with f3b5f1bebf0cad583f6eb13be366cf971eda18d1a26d6cf4d64636d89f8f1fd6 not found: ID does not exist"
Jul 29 22:19:52.557639 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.557616    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"]
Jul 29 22:19:52.564132 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.564088    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-eafc3c-8549f8cdf-94q2f"]
Jul 29 22:19:52.574823 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.574802    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"]
Jul 29 22:19:52.582012 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:52.581992    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-eafc3c-5db6ccb77d-9x5nk"]
Jul 29 22:19:53.306067 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:53.306033    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="282cac88-fac8-4361-a501-732e00725b8b" path="/var/lib/kubelet/pods/282cac88-fac8-4361-a501-732e00725b8b/volumes"
Jul 29 22:19:53.306634 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:53.306613    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" path="/var/lib/kubelet/pods/c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e/volumes"
Jul 29 22:19:56.622263 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:56.622220    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:19:57.855197 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:57.855136    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:57.956637 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:57.956594    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:58.217411 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:58.217371    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:58.318298 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:58.318262    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:58.502956 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:58.502875    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:19:58.727352 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:19:58.727319    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:20:02.300054 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300022    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"]
Jul 29 22:20:02.300509 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300492    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="282cac88-fac8-4361-a501-732e00725b8b" containerName="payload-processing"
Jul 29 22:20:02.300509 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300510    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="282cac88-fac8-4361-a501-732e00725b8b" containerName="payload-processing"
Jul 29 22:20:02.300586 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300532    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" containerName="maas-api"
Jul 29 22:20:02.300586 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300539    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" containerName="maas-api"
Jul 29 22:20:02.300672 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300603    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ea4e6e-e93a-4502-b5d6-dcbb9df59f1e" containerName="maas-api"
Jul 29 22:20:02.300672 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.300613    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="282cac88-fac8-4361-a501-732e00725b8b" containerName="payload-processing"
Jul 29 22:20:02.309185 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.309154    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:02.311531 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.311510    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:20:02.311531 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.311511    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxbps\""
Jul 29 22:20:02.312539 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.312517    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"]
Jul 29 22:20:02.438427 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.438384    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/153bf2bd-132c-40c4-b497-2c11320b1a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp\" (UID: \"153bf2bd-132c-40c4-b497-2c11320b1a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:02.539440 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.539401    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/153bf2bd-132c-40c4-b497-2c11320b1a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp\" (UID: \"153bf2bd-132c-40c4-b497-2c11320b1a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:02.540057 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.540036    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/153bf2bd-132c-40c4-b497-2c11320b1a35-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp\" (UID: \"153bf2bd-132c-40c4-b497-2c11320b1a35\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:02.621724 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.621629    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:02.766464 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:02.766432    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"]
Jul 29 22:20:02.768791 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:20:02.768760    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod153bf2bd_132c_40c4_b497_2c11320b1a35.slice/crio-96c3d74d2b47a07b33eca21dd4b74573ce82c580ae2d4002529c9b691c488dcf WatchSource:0}: Error finding container 96c3d74d2b47a07b33eca21dd4b74573ce82c580ae2d4002529c9b691c488dcf: Status 404 returned error can't find the container with id 96c3d74d2b47a07b33eca21dd4b74573ce82c580ae2d4002529c9b691c488dcf
Jul 29 22:20:03.605195 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:03.605134    2572 generic.go:356] "Generic (PLEG): container finished" podID="153bf2bd-132c-40c4-b497-2c11320b1a35" containerID="603fa1c9ca45fac195cba13bd235ea87d3745c80a20f61926b758c2155f69cda" exitCode=0
Jul 29 22:20:03.605623 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:03.605236    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp" event={"ID":"153bf2bd-132c-40c4-b497-2c11320b1a35","Type":"ContainerDied","Data":"603fa1c9ca45fac195cba13bd235ea87d3745c80a20f61926b758c2155f69cda"}
Jul 29 22:20:03.605623 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:03.605298    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp" event={"ID":"153bf2bd-132c-40c4-b497-2c11320b1a35","Type":"ContainerStarted","Data":"96c3d74d2b47a07b33eca21dd4b74573ce82c580ae2d4002529c9b691c488dcf"}
Jul 29 22:20:04.757184 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:04.757150    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:04.861051 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:04.861019    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/153bf2bd-132c-40c4-b497-2c11320b1a35-maas-api-service-ca\") pod \"153bf2bd-132c-40c4-b497-2c11320b1a35\" (UID: \"153bf2bd-132c-40c4-b497-2c11320b1a35\") "
Jul 29 22:20:04.861389 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:04.861368    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/153bf2bd-132c-40c4-b497-2c11320b1a35-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "153bf2bd-132c-40c4-b497-2c11320b1a35" (UID: "153bf2bd-132c-40c4-b497-2c11320b1a35"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:20:04.962097 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:04.962064    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/153bf2bd-132c-40c4-b497-2c11320b1a35-maas-api-service-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:05.618237 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:05.618138    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"
Jul 29 22:20:05.618237 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:05.618157    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp" event={"ID":"153bf2bd-132c-40c4-b497-2c11320b1a35","Type":"ContainerDied","Data":"96c3d74d2b47a07b33eca21dd4b74573ce82c580ae2d4002529c9b691c488dcf"}
Jul 29 22:20:05.618237 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:05.618216    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96c3d74d2b47a07b33eca21dd4b74573ce82c580ae2d4002529c9b691c488dcf"
Jul 29 22:20:17.336695 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:17.336661    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:20:17.923257 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:17.923223    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:20:18.096516 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:18.096472    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"]
Jul 29 22:20:18.101660 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:18.101628    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-ee9d15-797lp"]
Jul 29 22:20:19.304600 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:19.304565    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="153bf2bd-132c-40c4-b497-2c11320b1a35" path="/var/lib/kubelet/pods/153bf2bd-132c-40c4-b497-2c11320b1a35/volumes"
Jul 29 22:20:19.663633 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:19.663535    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:20:23.477877 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:23.477840    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"]
Jul 29 22:20:23.572123 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:23.572080    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:20:27.933397 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:27.933352    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:20:38.497746 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.497691    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j" podUID="a2415aa7-2d70-435a-9f4a-f395d39d3f07" containerName="main" containerID="cri-o://21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9" gracePeriod=45
Jul 29 22:20:38.773479 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.773453    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:20:38.821058 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.821022    2572 generic.go:356] "Generic (PLEG): container finished" podID="a2415aa7-2d70-435a-9f4a-f395d39d3f07" containerID="21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9" exitCode=0
Jul 29 22:20:38.821257 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.821080    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"
Jul 29 22:20:38.821257 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.821110    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j" event={"ID":"a2415aa7-2d70-435a-9f4a-f395d39d3f07","Type":"ContainerDied","Data":"21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9"}
Jul 29 22:20:38.821257 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.821163    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j" event={"ID":"a2415aa7-2d70-435a-9f4a-f395d39d3f07","Type":"ContainerDied","Data":"8ee01fb267acf7b239288836bf4b434463fa8b80bc4f381f448e5acd7272ca4b"}
Jul 29 22:20:38.821257 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.821205    2572 scope.go:117] "RemoveContainer" containerID="21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9"
Jul 29 22:20:38.834443 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.834420    2572 scope.go:117] "RemoveContainer" containerID="21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9"
Jul 29 22:20:38.834784 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:20:38.834759    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9\": container with ID starting with 21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9 not found: ID does not exist" containerID="21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9"
Jul 29 22:20:38.834852 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.834799    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9"} err="failed to get container status \"21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9\": rpc error: code = NotFound desc = could not find container \"21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9\": container with ID starting with 21fff2cb7b6fe3433247055bd7f16b063c6ceaf70b55f66ce223a2d5f3408da9 not found: ID does not exist"
Jul 29 22:20:38.892041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892005    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-dshm\") pod \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") "
Jul 29 22:20:38.892266 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892068    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-model-cache\") pod \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") "
Jul 29 22:20:38.892266 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892119    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tls-certs\") pod \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") "
Jul 29 22:20:38.892266 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892141    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tmp-dir\") pod \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") "
Jul 29 22:20:38.892266 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892236    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-home\") pod \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") "
Jul 29 22:20:38.892266 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892260    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf6fq\" (UniqueName: \"kubernetes.io/projected/a2415aa7-2d70-435a-9f4a-f395d39d3f07-kube-api-access-hf6fq\") pod \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\" (UID: \"a2415aa7-2d70-435a-9f4a-f395d39d3f07\") "
Jul 29 22:20:38.892813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892402    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2415aa7-2d70-435a-9f4a-f395d39d3f07" (UID: "a2415aa7-2d70-435a-9f4a-f395d39d3f07"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:20:38.892813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892452    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2415aa7-2d70-435a-9f4a-f395d39d3f07" (UID: "a2415aa7-2d70-435a-9f4a-f395d39d3f07"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:20:38.892813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892486    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-home" (OuterVolumeSpecName: "home") pod "a2415aa7-2d70-435a-9f4a-f395d39d3f07" (UID: "a2415aa7-2d70-435a-9f4a-f395d39d3f07"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:20:38.892813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892701    2572 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-model-cache\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:38.892813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892725    2572 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tmp-dir\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:38.892813 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.892738    2572 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-home\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:38.894437 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.894407    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-dshm" (OuterVolumeSpecName: "dshm") pod "a2415aa7-2d70-435a-9f4a-f395d39d3f07" (UID: "a2415aa7-2d70-435a-9f4a-f395d39d3f07"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:20:38.894546 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.894499    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2415aa7-2d70-435a-9f4a-f395d39d3f07" (UID: "a2415aa7-2d70-435a-9f4a-f395d39d3f07"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 22:20:38.894546 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.894515    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2415aa7-2d70-435a-9f4a-f395d39d3f07-kube-api-access-hf6fq" (OuterVolumeSpecName: "kube-api-access-hf6fq") pod "a2415aa7-2d70-435a-9f4a-f395d39d3f07" (UID: "a2415aa7-2d70-435a-9f4a-f395d39d3f07"). InnerVolumeSpecName "kube-api-access-hf6fq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:20:38.993964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.993930    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf6fq\" (UniqueName: \"kubernetes.io/projected/a2415aa7-2d70-435a-9f4a-f395d39d3f07-kube-api-access-hf6fq\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:38.993964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.993956    2572 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2415aa7-2d70-435a-9f4a-f395d39d3f07-dshm\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:38.993964 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:38.993966    2572 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2415aa7-2d70-435a-9f4a-f395d39d3f07-tls-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:20:39.146090 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:39.146046    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"]
Jul 29 22:20:39.150390 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:39.150355    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-ee9d15/ipp-route-ee9d15-kserve-6875777887-94s5j"]
Jul 29 22:20:39.305612 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:39.305571    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2415aa7-2d70-435a-9f4a-f395d39d3f07" path="/var/lib/kubelet/pods/a2415aa7-2d70-435a-9f4a-f395d39d3f07/volumes"
Jul 29 22:20:53.677760 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:53.677671    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:20:53.737571 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:53.737539    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:20:59.710849 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:20:59.710815    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:00.730156 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:00.730126    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:06.214474 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:06.214440    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:11.810531 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:11.810487    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:11.878731 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:11.878692    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:12.094355 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.094244    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"]
Jul 29 22:21:12.094560 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.094531    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy" containerID="cri-o://379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0" gracePeriod=30
Jul 29 22:21:12.115316 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.115290    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:12.367820 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.367725    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:12.461470 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461436    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"]
Jul 29 22:21:12.461878 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461865    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2415aa7-2d70-435a-9f4a-f395d39d3f07" containerName="main"
Jul 29 22:21:12.461926 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461880    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2415aa7-2d70-435a-9f4a-f395d39d3f07" containerName="main"
Jul 29 22:21:12.461926 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461905    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="153bf2bd-132c-40c4-b497-2c11320b1a35" containerName="revoke-keys"
Jul 29 22:21:12.461926 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461911    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="153bf2bd-132c-40c4-b497-2c11320b1a35" containerName="revoke-keys"
Jul 29 22:21:12.462029 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461979    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="153bf2bd-132c-40c4-b497-2c11320b1a35" containerName="revoke-keys"
Jul 29 22:21:12.462029 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.461989    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2415aa7-2d70-435a-9f4a-f395d39d3f07" containerName="main"
Jul 29 22:21:12.464716 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.464698    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:12.466873 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.466848    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:21:12.467008 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.466990    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxbps\""
Jul 29 22:21:12.472633 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.472560    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"]
Jul 29 22:21:12.517857 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.517821    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b9562d-e628-49a2-836d-2278b12bb0f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk\" (UID: \"03b9562d-e628-49a2-836d-2278b12bb0f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:12.618480 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.618386    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b9562d-e628-49a2-836d-2278b12bb0f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk\" (UID: \"03b9562d-e628-49a2-836d-2278b12bb0f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:12.619017 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.618995    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b9562d-e628-49a2-836d-2278b12bb0f2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk\" (UID: \"03b9562d-e628-49a2-836d-2278b12bb0f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:12.777800 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:12.777769    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:13.130563 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:13.130536    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"]
Jul 29 22:21:13.132736 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:21:13.132707    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b9562d_e628_49a2_836d_2278b12bb0f2.slice/crio-5bc3cfcee1344b9b80853ec04aa0beb573b57cb8791d8823b50e159f3ec4c394 WatchSource:0}: Error finding container 5bc3cfcee1344b9b80853ec04aa0beb573b57cb8791d8823b50e159f3ec4c394: Status 404 returned error can't find the container with id 5bc3cfcee1344b9b80853ec04aa0beb573b57cb8791d8823b50e159f3ec4c394
Jul 29 22:21:14.045222 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:14.045158    2572 generic.go:356] "Generic (PLEG): container finished" podID="03b9562d-e628-49a2-836d-2278b12bb0f2" containerID="6a904f857c1ebe40254fa822777bbf5a203ad5703d218bedcca85791d1ef8911" exitCode=0
Jul 29 22:21:14.045222 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:14.045209    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk" event={"ID":"03b9562d-e628-49a2-836d-2278b12bb0f2","Type":"ContainerDied","Data":"6a904f857c1ebe40254fa822777bbf5a203ad5703d218bedcca85791d1ef8911"}
Jul 29 22:21:14.045691 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:14.045278    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk" event={"ID":"03b9562d-e628-49a2-836d-2278b12bb0f2","Type":"ContainerStarted","Data":"5bc3cfcee1344b9b80853ec04aa0beb573b57cb8791d8823b50e159f3ec4c394"}
Jul 29 22:21:15.204303 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:15.204275    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:15.344758 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:15.344669    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b9562d-e628-49a2-836d-2278b12bb0f2-maas-api-service-ca\") pod \"03b9562d-e628-49a2-836d-2278b12bb0f2\" (UID: \"03b9562d-e628-49a2-836d-2278b12bb0f2\") "
Jul 29 22:21:15.345085 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:15.345065    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03b9562d-e628-49a2-836d-2278b12bb0f2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "03b9562d-e628-49a2-836d-2278b12bb0f2" (UID: "03b9562d-e628-49a2-836d-2278b12bb0f2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:21:15.446285 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:15.446245    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/03b9562d-e628-49a2-836d-2278b12bb0f2-maas-api-service-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:16.058456 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:16.058425    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"
Jul 29 22:21:16.058645 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:16.058458    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk" event={"ID":"03b9562d-e628-49a2-836d-2278b12bb0f2","Type":"ContainerDied","Data":"5bc3cfcee1344b9b80853ec04aa0beb573b57cb8791d8823b50e159f3ec4c394"}
Jul 29 22:21:16.058645 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:16.058506    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5bc3cfcee1344b9b80853ec04aa0beb573b57cb8791d8823b50e159f3ec4c394"
Jul 29 22:21:17.354478 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.354412    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:21:17.465231 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465196    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-podinfo\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465394 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465264    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istiod-ca-cert\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465394 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465288    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-data\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465394 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465311    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-token\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465394 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465331    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-credential-socket\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465394 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465369    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-certs\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465687 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465429    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-socket\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465687 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465497    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l65lx\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-kube-api-access-l65lx\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465687 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465539    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-envoy\") pod \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\" (UID: \"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7\") "
Jul 29 22:21:17.465862 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465673    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:21:17.465862 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465749    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:21:17.465862 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465834    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:21:17.465985 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.465890    2572 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-credential-socket\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.466276 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.466213    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:21:17.466423 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.466345    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-data" (OuterVolumeSpecName: "istio-data") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:21:17.467508 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.467480    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 22:21:17.467634 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.467616    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 22:21:17.467703 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.467672    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-token" (OuterVolumeSpecName: "istio-token") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:21:17.467790 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.467748    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-kube-api-access-l65lx" (OuterVolumeSpecName: "kube-api-access-l65lx") pod "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" (UID: "bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7"). InnerVolumeSpecName "kube-api-access-l65lx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:21:17.567278 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567246    2572 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-podinfo\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567278 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567271    2572 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istiod-ca-cert\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567278 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567280    2572 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-data\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567288    2572 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-token\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567298    2572 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-certs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567308    2572 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-workload-socket\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567319    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l65lx\" (UniqueName: \"kubernetes.io/projected/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-kube-api-access-l65lx\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:17.567507 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:17.567330    2572 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7-istio-envoy\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:18.072124 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.072087    2572 generic.go:356] "Generic (PLEG): container finished" podID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerID="379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0" exitCode=0
Jul 29 22:21:18.072334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.072156    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" event={"ID":"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7","Type":"ContainerDied","Data":"379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0"}
Jul 29 22:21:18.072334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.072257    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk" event={"ID":"bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7","Type":"ContainerDied","Data":"2b0b4057f26e0471ca65fbd590565c14440f8d4bc59cb732862428a493d0d06a"}
Jul 29 22:21:18.072334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.072157    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"
Jul 29 22:21:18.072334 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.072281    2572 scope.go:117] "RemoveContainer" containerID="379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0"
Jul 29 22:21:18.082883 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.082861    2572 scope.go:117] "RemoveContainer" containerID="a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"
Jul 29 22:21:18.092108 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.092064    2572 scope.go:117] "RemoveContainer" containerID="379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0"
Jul 29 22:21:18.092568 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:21:18.092521    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0\": container with ID starting with 379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0 not found: ID does not exist" containerID="379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0"
Jul 29 22:21:18.092685 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.092620    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0"} err="failed to get container status \"379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0\": rpc error: code = NotFound desc = could not find container \"379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0\": container with ID starting with 379243ac5aed2d038fa874bb08e30a8b06d7e8ac8993039fd344f84b301b3ec0 not found: ID does not exist"
Jul 29 22:21:18.092685 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.092652    2572 scope.go:117] "RemoveContainer" containerID="a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"
Jul 29 22:21:18.092992 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:21:18.092973    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484\": container with ID starting with a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484 not found: ID does not exist" containerID="a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"
Jul 29 22:21:18.093043 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.092999    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484"} err="failed to get container status \"a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484\": rpc error: code = NotFound desc = could not find container \"a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484\": container with ID starting with a504449fc0aa19058119baf5f05eaa1e5a497d7e5060c197797c1e77373cc484 not found: ID does not exist"
Jul 29 22:21:18.095279 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.095257    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"]
Jul 29 22:21:18.099109 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:18.099088    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-ee9d15-openshift-default-84c9b69948-c45qk"]
Jul 29 22:21:19.304320 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:19.304283    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" path="/var/lib/kubelet/pods/bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7/volumes"
Jul 29 22:21:23.069799 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.069760    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:23.146130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.146093    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:23.218831 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.218789    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:23.247793 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.247757    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"]
Jul 29 22:21:23.250009 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.249988    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-b26bbb-xcqmk"]
Jul 29 22:21:23.277713 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.277675    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:23.304453 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.304419    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03b9562d-e628-49a2-836d-2278b12bb0f2" path="/var/lib/kubelet/pods/03b9562d-e628-49a2-836d-2278b12bb0f2/volumes"
Jul 29 22:21:23.663885 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:23.663851    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:26.328371 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:26.328329    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:21:29.288145 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.288112    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:29.350948 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.350901    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:29.572555 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.572473    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:29.818394 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.818356    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:29.964876 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.964839    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"]
Jul 29 22:21:29.965335 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965322    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy"
Jul 29 22:21:29.965384 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965337    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy"
Jul 29 22:21:29.965384 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965346    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03b9562d-e628-49a2-836d-2278b12bb0f2" containerName="revoke-keys"
Jul 29 22:21:29.965384 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965352    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="03b9562d-e628-49a2-836d-2278b12bb0f2" containerName="revoke-keys"
Jul 29 22:21:29.965492 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965444    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy"
Jul 29 22:21:29.965492 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965453    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="03b9562d-e628-49a2-836d-2278b12bb0f2" containerName="revoke-keys"
Jul 29 22:21:29.965492 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.965468    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy"
Jul 29 22:21:29.970138 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.970121    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:29.972541 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.972489    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxbps\""
Jul 29 22:21:29.972697 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.972502    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:21:29.976884 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:29.976856    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"]
Jul 29 22:21:30.002701 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:30.002667    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1328578a-a2e3-40ec-bfae-f0df89818be7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z\" (UID: \"1328578a-a2e3-40ec-bfae-f0df89818be7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:30.103947 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:30.103906    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1328578a-a2e3-40ec-bfae-f0df89818be7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z\" (UID: \"1328578a-a2e3-40ec-bfae-f0df89818be7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:30.104574 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:30.104553    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1328578a-a2e3-40ec-bfae-f0df89818be7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z\" (UID: \"1328578a-a2e3-40ec-bfae-f0df89818be7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:30.283814 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:30.283784    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:30.431578 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:30.431547    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"]
Jul 29 22:21:30.433664 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:21:30.433634    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1328578a_a2e3_40ec_bfae_f0df89818be7.slice/crio-3c2a79b34092a42f649917ace46c02549a91663dde150491ba57a5fdc546b02f WatchSource:0}: Error finding container 3c2a79b34092a42f649917ace46c02549a91663dde150491ba57a5fdc546b02f: Status 404 returned error can't find the container with id 3c2a79b34092a42f649917ace46c02549a91663dde150491ba57a5fdc546b02f
Jul 29 22:21:31.152243 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:31.152134    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z" event={"ID":"1328578a-a2e3-40ec-bfae-f0df89818be7","Type":"ContainerStarted","Data":"a86838f1631df4c80d6d3043e0052b4c2e9cec95d7c9a9ced4b33670649b7071"}
Jul 29 22:21:31.152243 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:31.152213    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z" event={"ID":"1328578a-a2e3-40ec-bfae-f0df89818be7","Type":"ContainerStarted","Data":"3c2a79b34092a42f649917ace46c02549a91663dde150491ba57a5fdc546b02f"}
Jul 29 22:21:31.199689 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:31.199632    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z" podStartSLOduration=2.199613995 podStartE2EDuration="2.199613995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:21:31.198018955 +0000 UTC m=+3902.414796416" watchObservedRunningTime="2026-07-29 22:21:31.199613995 +0000 UTC m=+3902.416391484"
Jul 29 22:21:32.159878 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:32.159842    2572 generic.go:356] "Generic (PLEG): container finished" podID="1328578a-a2e3-40ec-bfae-f0df89818be7" containerID="a86838f1631df4c80d6d3043e0052b4c2e9cec95d7c9a9ced4b33670649b7071" exitCode=0
Jul 29 22:21:32.160316 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:32.159911    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z" event={"ID":"1328578a-a2e3-40ec-bfae-f0df89818be7","Type":"ContainerDied","Data":"a86838f1631df4c80d6d3043e0052b4c2e9cec95d7c9a9ced4b33670649b7071"}
Jul 29 22:21:33.310201 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:33.310160    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:33.332839 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:33.332813    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1328578a-a2e3-40ec-bfae-f0df89818be7-maas-api-service-ca\") pod \"1328578a-a2e3-40ec-bfae-f0df89818be7\" (UID: \"1328578a-a2e3-40ec-bfae-f0df89818be7\") "
Jul 29 22:21:33.333119 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:33.333092    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1328578a-a2e3-40ec-bfae-f0df89818be7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1328578a-a2e3-40ec-bfae-f0df89818be7" (UID: "1328578a-a2e3-40ec-bfae-f0df89818be7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:21:33.333283 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:33.333213    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1328578a-a2e3-40ec-bfae-f0df89818be7-maas-api-service-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:34.177137 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:34.177108    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"
Jul 29 22:21:34.177137 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:34.177097    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z" event={"ID":"1328578a-a2e3-40ec-bfae-f0df89818be7","Type":"ContainerDied","Data":"3c2a79b34092a42f649917ace46c02549a91663dde150491ba57a5fdc546b02f"}
Jul 29 22:21:34.177376 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:34.177189    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c2a79b34092a42f649917ace46c02549a91663dde150491ba57a5fdc546b02f"
Jul 29 22:21:40.704906 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.704871    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"]
Jul 29 22:21:40.713747 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.713720    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-c9eccd-29756055-v2xfw"]
Jul 29 22:21:40.716592 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.716566    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:40.740224 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.740199    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"]
Jul 29 22:21:40.740497 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.740442    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp" podUID="8d724c10-f9d0-49ff-b634-c2b4898016ce" containerName="payload-processing" containerID="cri-o://bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e" gracePeriod=30
Jul 29 22:21:40.814134 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.814100    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:40.893803 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.893766    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"]
Jul 29 22:21:40.895103 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.895074    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:40.897485 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.897463    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-c9eccd-6tf4z"]
Jul 29 22:21:40.961528 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:40.961437    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:41.012459 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.012434    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:21:41.104212 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.104185    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d724c10-f9d0-49ff-b634-c2b4898016ce-plugins-config-volume\") pod \"8d724c10-f9d0-49ff-b634-c2b4898016ce\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") "
Jul 29 22:21:41.104411 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.104242    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzfxs\" (UniqueName: \"kubernetes.io/projected/8d724c10-f9d0-49ff-b634-c2b4898016ce-kube-api-access-mzfxs\") pod \"8d724c10-f9d0-49ff-b634-c2b4898016ce\" (UID: \"8d724c10-f9d0-49ff-b634-c2b4898016ce\") "
Jul 29 22:21:41.104597 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.104572    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d724c10-f9d0-49ff-b634-c2b4898016ce-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8d724c10-f9d0-49ff-b634-c2b4898016ce" (UID: "8d724c10-f9d0-49ff-b634-c2b4898016ce"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:21:41.106411 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.106389    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d724c10-f9d0-49ff-b634-c2b4898016ce-kube-api-access-mzfxs" (OuterVolumeSpecName: "kube-api-access-mzfxs") pod "8d724c10-f9d0-49ff-b634-c2b4898016ce" (UID: "8d724c10-f9d0-49ff-b634-c2b4898016ce"). InnerVolumeSpecName "kube-api-access-mzfxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:21:41.206050 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.206015    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d724c10-f9d0-49ff-b634-c2b4898016ce-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:41.206050 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.206044    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzfxs\" (UniqueName: \"kubernetes.io/projected/8d724c10-f9d0-49ff-b634-c2b4898016ce-kube-api-access-mzfxs\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:41.218328 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.218253    2572 generic.go:356] "Generic (PLEG): container finished" podID="8d724c10-f9d0-49ff-b634-c2b4898016ce" containerID="bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e" exitCode=0
Jul 29 22:21:41.218328 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.218300    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"
Jul 29 22:21:41.218492 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.218341    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp" event={"ID":"8d724c10-f9d0-49ff-b634-c2b4898016ce","Type":"ContainerDied","Data":"bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e"}
Jul 29 22:21:41.218492 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.218394    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp" event={"ID":"8d724c10-f9d0-49ff-b634-c2b4898016ce","Type":"ContainerDied","Data":"ece9766478dd0b1fc1eb082671ff97f10ba3b59a9c4fb2783ed111579e996fb7"}
Jul 29 22:21:41.218492 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.218416    2572 scope.go:117] "RemoveContainer" containerID="bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e"
Jul 29 22:21:41.228522 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.228504    2572 scope.go:117] "RemoveContainer" containerID="bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e"
Jul 29 22:21:41.228748 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:21:41.228731    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e\": container with ID starting with bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e not found: ID does not exist" containerID="bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e"
Jul 29 22:21:41.228814 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.228755    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e"} err="failed to get container status \"bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e\": rpc error: code = NotFound desc = could not find container \"bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e\": container with ID starting with bdbde6b39e9a529674955176177772fce27358da50e927f668070f37ff0d9c5e not found: ID does not exist"
Jul 29 22:21:41.251684 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.251658    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"]
Jul 29 22:21:41.258067 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.258036    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-c9eccd-56b7486894-8f4xp"]
Jul 29 22:21:41.311062 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.311029    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1328578a-a2e3-40ec-bfae-f0df89818be7" path="/var/lib/kubelet/pods/1328578a-a2e3-40ec-bfae-f0df89818be7/volumes"
Jul 29 22:21:41.311333 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.311319    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36eb5a73-5887-4adc-bd22-0fe4690a9de3" path="/var/lib/kubelet/pods/36eb5a73-5887-4adc-bd22-0fe4690a9de3/volumes"
Jul 29 22:21:41.311603 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.311592    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d724c10-f9d0-49ff-b634-c2b4898016ce" path="/var/lib/kubelet/pods/8d724c10-f9d0-49ff-b634-c2b4898016ce/volumes"
Jul 29 22:21:41.317164 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:41.317136    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:43.922976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:43.922940    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:21:47.420752 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:47.420714    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:47.502139 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:47.502105    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:47.723620 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:47.723581    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:47.961077 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:47.961044    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:48.078235 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078143    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"]
Jul 29 22:21:48.078563 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078550    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy"
Jul 29 22:21:48.078614 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078565    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf2abd8-fc2a-4053-b420-b1cc0a96e0b7" containerName="istio-proxy"
Jul 29 22:21:48.078614 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078580    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d724c10-f9d0-49ff-b634-c2b4898016ce" containerName="payload-processing"
Jul 29 22:21:48.078614 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078586    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d724c10-f9d0-49ff-b634-c2b4898016ce" containerName="payload-processing"
Jul 29 22:21:48.078614 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078599    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1328578a-a2e3-40ec-bfae-f0df89818be7" containerName="revoke-keys"
Jul 29 22:21:48.078614 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078604    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="1328578a-a2e3-40ec-bfae-f0df89818be7" containerName="revoke-keys"
Jul 29 22:21:48.078762 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078672    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="1328578a-a2e3-40ec-bfae-f0df89818be7" containerName="revoke-keys"
Jul 29 22:21:48.078762 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.078681    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d724c10-f9d0-49ff-b634-c2b4898016ce" containerName="payload-processing"
Jul 29 22:21:48.081859 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.081842    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:48.084118 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.084099    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 22:21:48.084219 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.084152    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hxbps\""
Jul 29 22:21:48.088910 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.088883    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"]
Jul 29 22:21:48.179075 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.179028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5\" (UID: \"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:48.279986 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.279936    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5\" (UID: \"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:48.280653 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.280631    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5\" (UID: \"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:48.394096 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.394005    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:48.533982 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:48.533956    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"]
Jul 29 22:21:48.535983 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:21:48.535951    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8ac4eb_10dd_4c1e_aa8e_59a87a7373a6.slice/crio-92b34293fce8a5b9ed7d4357f6ac233627db05ff59855811b4d8cc1706863d2a WatchSource:0}: Error finding container 92b34293fce8a5b9ed7d4357f6ac233627db05ff59855811b4d8cc1706863d2a: Status 404 returned error can't find the container with id 92b34293fce8a5b9ed7d4357f6ac233627db05ff59855811b4d8cc1706863d2a
Jul 29 22:21:49.263910 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:49.263873    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5" event={"ID":"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6","Type":"ContainerStarted","Data":"2a1513b5b6bc23a99110adc3082fc89419f0a456856ae4aa7a8b281bc13836c0"}
Jul 29 22:21:49.264121 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:49.263936    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5" event={"ID":"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6","Type":"ContainerStarted","Data":"92b34293fce8a5b9ed7d4357f6ac233627db05ff59855811b4d8cc1706863d2a"}
Jul 29 22:21:49.281041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:49.280996    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5" podStartSLOduration=1.280979444 podStartE2EDuration="1.280979444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:21:49.278883591 +0000 UTC m=+3920.495661055" watchObservedRunningTime="2026-07-29 22:21:49.280979444 +0000 UTC m=+3920.497756905"
Jul 29 22:21:49.515726 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:49.515628    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:50.271247 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:50.271216    2572 generic.go:356] "Generic (PLEG): container finished" podID="0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6" containerID="2a1513b5b6bc23a99110adc3082fc89419f0a456856ae4aa7a8b281bc13836c0" exitCode=0
Jul 29 22:21:50.271640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:50.271304    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5" event={"ID":"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6","Type":"ContainerDied","Data":"2a1513b5b6bc23a99110adc3082fc89419f0a456856ae4aa7a8b281bc13836c0"}
Jul 29 22:21:51.419120 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:51.419086    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:51.511535 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:51.511496    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6-maas-api-service-ca\") pod \"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6\" (UID: \"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6\") "
Jul 29 22:21:51.511922 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:51.511889    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6" (UID: "0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:21:51.612652 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:51.612548    2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6-maas-api-service-ca\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:52.285626 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:52.285594    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"
Jul 29 22:21:52.285626 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:52.285611    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5" event={"ID":"0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6","Type":"ContainerDied","Data":"92b34293fce8a5b9ed7d4357f6ac233627db05ff59855811b4d8cc1706863d2a"}
Jul 29 22:21:52.285867 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:52.285664    2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92b34293fce8a5b9ed7d4357f6ac233627db05ff59855811b4d8cc1706863d2a"
Jul 29 22:21:58.633519 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.633483    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:58.634905 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.634884    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"]
Jul 29 22:21:58.639463 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.639437    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-ca895e-29756055-gtx22"]
Jul 29 22:21:58.683326 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.683291    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"]
Jul 29 22:21:58.683531 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.683506    2572 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps" podUID="82d7826d-1a4d-4bfc-a71f-899088479944" containerName="payload-pre-processing" containerID="cri-o://6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88" gracePeriod=30
Jul 29 22:21:58.732827 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.732799    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:58.837465 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.837426    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:58.848926 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.848897    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"]
Jul 29 22:21:58.851737 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.851711    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-ca895e-82qk5"]
Jul 29 22:21:58.905013 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.904932    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:58.936576 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:58.936552    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:21:59.086611 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.086577    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhdnd\" (UniqueName: \"kubernetes.io/projected/82d7826d-1a4d-4bfc-a71f-899088479944-kube-api-access-zhdnd\") pod \"82d7826d-1a4d-4bfc-a71f-899088479944\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") "
Jul 29 22:21:59.086799 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.086754    2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d7826d-1a4d-4bfc-a71f-899088479944-plugins-config-volume\") pod \"82d7826d-1a4d-4bfc-a71f-899088479944\" (UID: \"82d7826d-1a4d-4bfc-a71f-899088479944\") "
Jul 29 22:21:59.087127 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.087100    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82d7826d-1a4d-4bfc-a71f-899088479944-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82d7826d-1a4d-4bfc-a71f-899088479944" (UID: "82d7826d-1a4d-4bfc-a71f-899088479944"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 22:21:59.088644 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.088622    2572 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d7826d-1a4d-4bfc-a71f-899088479944-kube-api-access-zhdnd" (OuterVolumeSpecName: "kube-api-access-zhdnd") pod "82d7826d-1a4d-4bfc-a71f-899088479944" (UID: "82d7826d-1a4d-4bfc-a71f-899088479944"). InnerVolumeSpecName "kube-api-access-zhdnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 22:21:59.188085 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.188034    2572 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82d7826d-1a4d-4bfc-a71f-899088479944-plugins-config-volume\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:59.188085 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.188082    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhdnd\" (UniqueName: \"kubernetes.io/projected/82d7826d-1a4d-4bfc-a71f-899088479944-kube-api-access-zhdnd\") on node \"ip-10-0-129-242.ec2.internal\" DevicePath \"\""
Jul 29 22:21:59.256395 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.256361    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:21:59.305482 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.305449    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6" path="/var/lib/kubelet/pods/0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6/volumes"
Jul 29 22:21:59.305768 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.305754    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2" path="/var/lib/kubelet/pods/ae16bbba-e25d-4bd3-ab19-62c77d9eaeb2/volumes"
Jul 29 22:21:59.328405 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.328368    2572 generic.go:356] "Generic (PLEG): container finished" podID="82d7826d-1a4d-4bfc-a71f-899088479944" containerID="6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88" exitCode=0
Jul 29 22:21:59.328582 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.328425    2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"
Jul 29 22:21:59.328582 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.328453    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps" event={"ID":"82d7826d-1a4d-4bfc-a71f-899088479944","Type":"ContainerDied","Data":"6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88"}
Jul 29 22:21:59.328582 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.328530    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps" event={"ID":"82d7826d-1a4d-4bfc-a71f-899088479944","Type":"ContainerDied","Data":"ed3a305db7717fe2319c676714ede2e04f4de188c3302f4a59e7e924e458b18b"}
Jul 29 22:21:59.328582 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.328553    2572 scope.go:117] "RemoveContainer" containerID="6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88"
Jul 29 22:21:59.337931 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.337909    2572 scope.go:117] "RemoveContainer" containerID="6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88"
Jul 29 22:21:59.338235 ip-10-0-129-242 kubenswrapper[2572]: E0729 22:21:59.338215    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88\": container with ID starting with 6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88 not found: ID does not exist" containerID="6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88"
Jul 29 22:21:59.338285 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.338246    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88"} err="failed to get container status \"6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88\": rpc error: code = NotFound desc = could not find container \"6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88\": container with ID starting with 6e475f90b0e0202dec3c2dcac2de299047cb18fc4bf733fc8c1bb66444e04c88 not found: ID does not exist"
Jul 29 22:21:59.348256 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.348227    2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"]
Jul 29 22:21:59.353058 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:21:59.353034    2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-ca895e-7dddd58c68-m6cps"]
Jul 29 22:22:01.304319 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:01.304289    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82d7826d-1a4d-4bfc-a71f-899088479944" path="/var/lib/kubelet/pods/82d7826d-1a4d-4bfc-a71f-899088479944/volumes"
Jul 29 22:22:01.530436 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:01.530401    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-b4szp"]
Jul 29 22:22:05.234281 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:05.234230    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:22:05.299947 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:05.299911    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:22:05.522267 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:05.522145    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:22:05.770222 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:05.770188    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:22:06.204593 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:06.204563    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57b9dd47c6-qsw72_f78fb90c-a02b-488c-be41-142a87e9fb6d/authorino/0.log"
Jul 29 22:22:13.243633 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:13.243605    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mpcf4_38f5c933-cd6a-40ef-9a57-5d378f513282/server/0.log"
Jul 29 22:22:13.366721 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:13.366688    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pffpc_58cb8084-7ecc-40ab-a467-ca14d2e67719/manager/0.log"
Jul 29 22:22:13.491310 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:13.491281    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b55745745-gwjsn_8c7f984e-894a-4f5b-9419-b16627818310/manager/0.log"
Jul 29 22:22:13.504972 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:13.504913    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b55745745-gwjsn_8c7f984e-894a-4f5b-9419-b16627818310/copy-manifests/0.log"
Jul 29 22:22:14.496794 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:14.496761    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29756055-2gq5l_236bba51-a700-4bf9-87bb-8c29e7c526eb/cleanup/0.log"
Jul 29 22:22:15.433030 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.432995    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd_fad08190-5525-494d-917b-cb5a39c4b510/util/0.log"
Jul 29 22:22:15.440346 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.440315    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd_fad08190-5525-494d-917b-cb5a39c4b510/pull/0.log"
Jul 29 22:22:15.447495 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.447471    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd_fad08190-5525-494d-917b-cb5a39c4b510/extract/0.log"
Jul 29 22:22:15.563852 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.563821    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77_1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc/util/0.log"
Jul 29 22:22:15.569820 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.569793    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77_1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc/pull/0.log"
Jul 29 22:22:15.575391 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.575367    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77_1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc/extract/0.log"
Jul 29 22:22:15.689894 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.689816    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g_8cb15346-9fdf-4bf4-80dd-5b0f08dac116/util/0.log"
Jul 29 22:22:15.695893 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.695867    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g_8cb15346-9fdf-4bf4-80dd-5b0f08dac116/pull/0.log"
Jul 29 22:22:15.701727 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.701698    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g_8cb15346-9fdf-4bf4-80dd-5b0f08dac116/extract/0.log"
Jul 29 22:22:15.819541 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.819510    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x_77c2f682-844f-4e0c-83be-f66b9deb82da/util/0.log"
Jul 29 22:22:15.825743 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.825708    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x_77c2f682-844f-4e0c-83be-f66b9deb82da/pull/0.log"
Jul 29 22:22:15.831120 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.831099    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x_77c2f682-844f-4e0c-83be-f66b9deb82da/extract/0.log"
Jul 29 22:22:15.960986 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:15.960879    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57b9dd47c6-qsw72_f78fb90c-a02b-488c-be41-142a87e9fb6d/authorino/0.log"
Jul 29 22:22:16.078490 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.078460    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7bvsf_89fb5a08-2220-444e-a86a-7c57191ef16d/manager/0.log"
Jul 29 22:22:16.194233 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.194205    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-df5tk_4425f501-704a-4593-8dbc-52ef60091af9/manager/0.log"
Jul 29 22:22:16.310378 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.310297    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9sl6j_b9e6aa38-fe24-4e94-b741-25bfaa10e60c/kuadrant-console-plugin/0.log"
Jul 29 22:22:16.434487 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.434453    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-cgp8k_db724bb6-3ffe-4cac-a364-569c44b0f199/registry-server/0.log"
Jul 29 22:22:16.555129 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.555103    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rgsh4_9c1c83f8-454c-4820-8da9-f5f3fd598708/manager/0.log"
Jul 29 22:22:16.666895 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.666766    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b4szp_897dd401-eb41-4c00-be54-2d5427c7ee55/limitador/0.log"
Jul 29 22:22:16.784640 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:16.784614    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9bgn6_b329228a-d4b1-41e5-9776-064d254597eb/manager/0.log"
Jul 29 22:22:17.502506 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:17.502474    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787cd99557-fzfmn_dbb736d7-8d40-4c69-990b-a3db196259aa/kube-auth-proxy/0.log"
Jul 29 22:22:18.591781 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:18.591750    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vljm9_cbb85914-239b-4006-9495-aaa5f297e1f2/main/0.log"
Jul 29 22:22:18.709187 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:18.709149    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ffmc2p_a7e1eb19-ab90-41f8-82e3-9ea908290774/main/0.log"
Jul 29 22:22:18.944985 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:18.944960    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-7p25z_987ea528-3f26-40ea-9851-794a5d9aa16b/main/0.log"
Jul 29 22:22:24.800067 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.799975    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jqkkn/must-gather-4ln4f"]
Jul 29 22:22:24.800976 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.800951    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d7826d-1a4d-4bfc-a71f-899088479944" containerName="payload-pre-processing"
Jul 29 22:22:24.801041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.800980    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d7826d-1a4d-4bfc-a71f-899088479944" containerName="payload-pre-processing"
Jul 29 22:22:24.801041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.801023    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6" containerName="revoke-keys"
Jul 29 22:22:24.801041 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.801032    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6" containerName="revoke-keys"
Jul 29 22:22:24.801250 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.801235    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d7826d-1a4d-4bfc-a71f-899088479944" containerName="payload-pre-processing"
Jul 29 22:22:24.801305 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.801268    2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c8ac4eb-10dd-4c1e-aa8e-59a87a7373a6" containerName="revoke-keys"
Jul 29 22:22:24.809584 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.809554    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:24.812646 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.812619    2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jqkkn\"/\"default-dockercfg-vhgbw\""
Jul 29 22:22:24.813066 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.813039    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jqkkn/must-gather-4ln4f"]
Jul 29 22:22:24.813206 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.813122    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jqkkn\"/\"openshift-service-ca.crt\""
Jul 29 22:22:24.813482 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.813465    2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jqkkn\"/\"kube-root-ca.crt\""
Jul 29 22:22:24.818061 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.818032    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:22:24.882966 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.882929    2572 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log" limit=1024
Jul 29 22:22:24.926810 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.926782    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f124020e-75c6-47b5-97b3-ebc66c911e4a-must-gather-output\") pod \"must-gather-4ln4f\" (UID: \"f124020e-75c6-47b5-97b3-ebc66c911e4a\") " pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:24.926933 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:24.926909    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvm4q\" (UniqueName: \"kubernetes.io/projected/f124020e-75c6-47b5-97b3-ebc66c911e4a-kube-api-access-lvm4q\") pod \"must-gather-4ln4f\" (UID: \"f124020e-75c6-47b5-97b3-ebc66c911e4a\") " pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:25.027619 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.027587    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvm4q\" (UniqueName: \"kubernetes.io/projected/f124020e-75c6-47b5-97b3-ebc66c911e4a-kube-api-access-lvm4q\") pod \"must-gather-4ln4f\" (UID: \"f124020e-75c6-47b5-97b3-ebc66c911e4a\") " pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:25.027777 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.027636    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f124020e-75c6-47b5-97b3-ebc66c911e4a-must-gather-output\") pod \"must-gather-4ln4f\" (UID: \"f124020e-75c6-47b5-97b3-ebc66c911e4a\") " pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:25.027983 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.027966    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f124020e-75c6-47b5-97b3-ebc66c911e4a-must-gather-output\") pod \"must-gather-4ln4f\" (UID: \"f124020e-75c6-47b5-97b3-ebc66c911e4a\") " pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:25.035803 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.035779    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvm4q\" (UniqueName: \"kubernetes.io/projected/f124020e-75c6-47b5-97b3-ebc66c911e4a-kube-api-access-lvm4q\") pod \"must-gather-4ln4f\" (UID: \"f124020e-75c6-47b5-97b3-ebc66c911e4a\") " pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:25.121210 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.121101    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jqkkn/must-gather-4ln4f"
Jul 29 22:22:25.252953 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.252929    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jqkkn/must-gather-4ln4f"]
Jul 29 22:22:25.254774 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:22:25.254739    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf124020e_75c6_47b5_97b3_ebc66c911e4a.slice/crio-ea49975ed6bb712fbe2c2643c2f56190e79ff69af39eea822302ec67abdf94ad WatchSource:0}: Error finding container ea49975ed6bb712fbe2c2643c2f56190e79ff69af39eea822302ec67abdf94ad: Status 404 returned error can't find the container with id ea49975ed6bb712fbe2c2643c2f56190e79ff69af39eea822302ec67abdf94ad
Jul 29 22:22:25.259206 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.258450    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 22:22:25.493547 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:25.493511    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqkkn/must-gather-4ln4f" event={"ID":"f124020e-75c6-47b5-97b3-ebc66c911e4a","Type":"ContainerStarted","Data":"ea49975ed6bb712fbe2c2643c2f56190e79ff69af39eea822302ec67abdf94ad"}
Jul 29 22:22:26.504958 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:26.504921    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqkkn/must-gather-4ln4f" event={"ID":"f124020e-75c6-47b5-97b3-ebc66c911e4a","Type":"ContainerStarted","Data":"8a5a6b1f3f980249968367618674f9a82c64c1b445451d421a06de64f44c8515"}
Jul 29 22:22:26.505439 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:26.504970    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqkkn/must-gather-4ln4f" event={"ID":"f124020e-75c6-47b5-97b3-ebc66c911e4a","Type":"ContainerStarted","Data":"63234635f92c4ab9e34e6de5f9d5f0480197f1e7919bac98151a307111b935a9"}
Jul 29 22:22:26.522244 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:26.522184    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jqkkn/must-gather-4ln4f" podStartSLOduration=1.7201696229999999 podStartE2EDuration="2.522152971s" totalImagesPullingTime="801.983348ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:22:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:22:26.521140786 +0000 UTC m=+3957.737918248" watchObservedRunningTime="2026-07-29 22:22:26.522152971 +0000 UTC m=+3957.738930462"
Jul 29 22:22:27.878291 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:27.878256    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f7ghg_3921626f-33f3-4c92-bdbe-c69a9ae6e54a/global-pull-secret-syncer/0.log"
Jul 29 22:22:27.997643 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:27.997604    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5vkp2_5c5aa1de-e609-490d-80b4-0104d630b0f4/konnectivity-agent/0.log"
Jul 29 22:22:28.128105 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:28.128074    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-242.ec2.internal_bff06bf76c4dee0b9650b26a17b25f48/haproxy/0.log"
Jul 29 22:22:29.940695 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:29.940656    2572 scope.go:117] "RemoveContainer" containerID="f14cfe4a3f0ac059b3aa951f17313d53fed2db10755de6e50a34e18ff07ccd76"
Jul 29 22:22:29.954726 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:29.954696    2572 scope.go:117] "RemoveContainer" containerID="2ec1763b59d579afe99469c79923c7ee1b344b90215d3b3b9b981596f0394eb7"
Jul 29 22:22:32.068410 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.068372    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd_fad08190-5525-494d-917b-cb5a39c4b510/extract/0.log"
Jul 29 22:22:32.095557 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.095496    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd_fad08190-5525-494d-917b-cb5a39c4b510/util/0.log"
Jul 29 22:22:32.120415 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.120316    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599khhd_fad08190-5525-494d-917b-cb5a39c4b510/pull/0.log"
Jul 29 22:22:32.149381 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.149349    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77_1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc/extract/0.log"
Jul 29 22:22:32.170721 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.170691    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77_1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc/util/0.log"
Jul 29 22:22:32.193405 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.193379    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e066j77_1a7b3c7b-d5a6-454a-bd7c-8da8e65111bc/pull/0.log"
Jul 29 22:22:32.230234 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.230201    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g_8cb15346-9fdf-4bf4-80dd-5b0f08dac116/extract/0.log"
Jul 29 22:22:32.270463 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.270422    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g_8cb15346-9fdf-4bf4-80dd-5b0f08dac116/util/0.log"
Jul 29 22:22:32.296973 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.296941    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n9j8g_8cb15346-9fdf-4bf4-80dd-5b0f08dac116/pull/0.log"
Jul 29 22:22:32.336956 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.336883    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x_77c2f682-844f-4e0c-83be-f66b9deb82da/extract/0.log"
Jul 29 22:22:32.361518 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.361406    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x_77c2f682-844f-4e0c-83be-f66b9deb82da/util/0.log"
Jul 29 22:22:32.401469 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:32.401432    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1d8h6x_77c2f682-844f-4e0c-83be-f66b9deb82da/pull/0.log"
Jul 29 22:22:33.067243 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.067211    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57b9dd47c6-qsw72_f78fb90c-a02b-488c-be41-142a87e9fb6d/authorino/0.log"
Jul 29 22:22:33.105926 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.105885    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7bvsf_89fb5a08-2220-444e-a86a-7c57191ef16d/manager/0.log"
Jul 29 22:22:33.139364 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.139330    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-df5tk_4425f501-704a-4593-8dbc-52ef60091af9/manager/0.log"
Jul 29 22:22:33.177184 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.177136    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9sl6j_b9e6aa38-fe24-4e94-b741-25bfaa10e60c/kuadrant-console-plugin/0.log"
Jul 29 22:22:33.225801 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.225441    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-cgp8k_db724bb6-3ffe-4cac-a364-569c44b0f199/registry-server/0.log"
Jul 29 22:22:33.364921 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.364824    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rgsh4_9c1c83f8-454c-4820-8da9-f5f3fd598708/manager/0.log"
Jul 29 22:22:33.396324 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.396296    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-b4szp_897dd401-eb41-4c00-be54-2d5427c7ee55/limitador/0.log"
Jul 29 22:22:33.518186 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:33.518129    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-9bgn6_b329228a-d4b1-41e5-9776-064d254597eb/manager/0.log"
Jul 29 22:22:35.524961 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:35.524917    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-8vhcv_45e75ef5-8b16-4b9e-9715-d50c47da09f0/cluster-monitoring-operator/0.log"
Jul 29 22:22:35.718189 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:35.718149    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7fhrf_e18afab2-c123-4619-9e7d-e7962a72b274/node-exporter/0.log"
Jul 29 22:22:35.741111 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:35.741082    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7fhrf_e18afab2-c123-4619-9e7d-e7962a72b274/kube-rbac-proxy/0.log"
Jul 29 22:22:35.766773 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:35.766749    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7fhrf_e18afab2-c123-4619-9e7d-e7962a72b274/init-textfile/0.log"
Jul 29 22:22:35.968461 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:35.968426    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-xldn5_1440b53f-b05b-4f72-8747-601fbb1c0556/kube-rbac-proxy-main/0.log"
Jul 29 22:22:35.993255 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:35.993221    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-xldn5_1440b53f-b05b-4f72-8747-601fbb1c0556/kube-rbac-proxy-self/0.log"
Jul 29 22:22:36.015716 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.015685    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-xldn5_1440b53f-b05b-4f72-8747-601fbb1c0556/openshift-state-metrics/0.log"
Jul 29 22:22:36.302817 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.302732    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-gwxwx_63db8829-79ba-4be0-9190-1ffeff599c2e/prometheus-operator/0.log"
Jul 29 22:22:36.325002 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.324973    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-gwxwx_63db8829-79ba-4be0-9190-1ffeff599c2e/kube-rbac-proxy/0.log"
Jul 29 22:22:36.355137 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.355103    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-xkmh9_ee7b230f-8685-4ca2-b382-fdb37b89ed14/prometheus-operator-admission-webhook/0.log"
Jul 29 22:22:36.476654 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.476616    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55c45d54bc-w62kp_4a866806-4c3b-4de1-9eaf-fe12e47ee327/thanos-query/0.log"
Jul 29 22:22:36.500613 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.500579    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55c45d54bc-w62kp_4a866806-4c3b-4de1-9eaf-fe12e47ee327/kube-rbac-proxy-web/0.log"
Jul 29 22:22:36.522367 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.522335    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55c45d54bc-w62kp_4a866806-4c3b-4de1-9eaf-fe12e47ee327/kube-rbac-proxy/0.log"
Jul 29 22:22:36.544398 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.544368    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55c45d54bc-w62kp_4a866806-4c3b-4de1-9eaf-fe12e47ee327/prom-label-proxy/0.log"
Jul 29 22:22:36.572082 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.571998    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55c45d54bc-w62kp_4a866806-4c3b-4de1-9eaf-fe12e47ee327/kube-rbac-proxy-rules/0.log"
Jul 29 22:22:36.606885 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:36.606851    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55c45d54bc-w62kp_4a866806-4c3b-4de1-9eaf-fe12e47ee327/kube-rbac-proxy-metrics/0.log"
Jul 29 22:22:37.784014 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:37.783988    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-q4csj_416db6ee-0be2-4e75-95c7-bd3d2da60d8a/networking-console-plugin/0.log"
Jul 29 22:22:38.342835 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.342784    2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"]
Jul 29 22:22:38.352724 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.352686    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.357916 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.357877    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"]
Jul 29 22:22:38.506588 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.506552    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-podres\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.506769 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.506644    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbq95\" (UniqueName: \"kubernetes.io/projected/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-kube-api-access-kbq95\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.506769 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.506681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-lib-modules\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.506769 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.506724    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-sys\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.506889 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.506772    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-proc\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.608210 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.608110    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-sys\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.608577 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.608556    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-proc\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.608816 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.608801    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-podres\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.609106 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.609090    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbq95\" (UniqueName: \"kubernetes.io/projected/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-kube-api-access-kbq95\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.609592 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.609569    2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-lib-modules\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.610199 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.608997    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-podres\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.610309 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.608713    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-proc\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.610309 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.608477    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-sys\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.610309 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.609833    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-lib-modules\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.618384 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.618362    2572 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbq95\" (UniqueName: \"kubernetes.io/projected/e41e69d8-ec4e-46a6-bddb-b6dfa5840eac-kube-api-access-kbq95\") pod \"perf-node-gather-daemonset-vwq6p\" (UID: \"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac\") " pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.674234 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.674195    2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:38.864943 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:38.864405    2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"]
Jul 29 22:22:38.867942 ip-10-0-129-242 kubenswrapper[2572]: W0729 22:22:38.867888    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode41e69d8_ec4e_46a6_bddb_b6dfa5840eac.slice/crio-4b36325199d91032b6937ae823dfa2d92a185c33d84ccfb3fb998cba01b9ac49 WatchSource:0}: Error finding container 4b36325199d91032b6937ae823dfa2d92a185c33d84ccfb3fb998cba01b9ac49: Status 404 returned error can't find the container with id 4b36325199d91032b6937ae823dfa2d92a185c33d84ccfb3fb998cba01b9ac49
Jul 29 22:22:39.628249 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:39.628206    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p" event={"ID":"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac","Type":"ContainerStarted","Data":"b444f00198821034d0dfddcbfc6ab996f17e478963934f42452ac9f7cdb8bdfd"}
Jul 29 22:22:39.628484 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:39.628296    2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p" event={"ID":"e41e69d8-ec4e-46a6-bddb-b6dfa5840eac","Type":"ContainerStarted","Data":"4b36325199d91032b6937ae823dfa2d92a185c33d84ccfb3fb998cba01b9ac49"}
Jul 29 22:22:39.629028 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:39.629000    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:39.649113 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:39.649053    2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p" podStartSLOduration=1.649035094 podStartE2EDuration="1.649035094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:22:39.646751513 +0000 UTC m=+3970.863528976" watchObservedRunningTime="2026-07-29 22:22:39.649035094 +0000 UTC m=+3970.865812555"
Jul 29 22:22:40.679736 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:40.679703    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m6l9g_956e2318-8d5c-46db-9e95-5f0ff311393b/dns/0.log"
Jul 29 22:22:40.703835 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:40.703806    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m6l9g_956e2318-8d5c-46db-9e95-5f0ff311393b/kube-rbac-proxy/0.log"
Jul 29 22:22:40.728563 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:40.728534    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4j287_d0679383-8d53-4278-958c-b66a18435ce0/dns-node-resolver/0.log"
Jul 29 22:22:41.298022 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:41.297978    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5bbd458499-cw25b_d374fdb0-fba8-4a86-8516-489a5c1249a0/registry/0.log"
Jul 29 22:22:41.345588 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:41.345552    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qsc5x_c54c0b98-fe67-4c62-9663-7a21dccf869b/node-ca/0.log"
Jul 29 22:22:42.541664 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:42.541629    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787cd99557-fzfmn_dbb736d7-8d40-4c69-990b-a3db196259aa/kube-auth-proxy/0.log"
Jul 29 22:22:43.551067 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:43.551034    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-p6jlr_ff96152c-cdba-4425-98b2-3b13d935763e/serve-healthcheck-canary/0.log"
Jul 29 22:22:44.086226 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:44.086186    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zd96v_ddf60a13-3a1c-41fa-8212-6762ab9a37c6/insights-operator/0.log"
Jul 29 22:22:44.087130 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:44.087098    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-zd96v_ddf60a13-3a1c-41fa-8212-6762ab9a37c6/insights-operator/1.log"
Jul 29 22:22:44.251946 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:44.251906    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmmvr_b6dc41d9-5eaa-482c-822a-b9e18c2f6b04/kube-rbac-proxy/0.log"
Jul 29 22:22:44.273671 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:44.273643    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmmvr_b6dc41d9-5eaa-482c-822a-b9e18c2f6b04/exporter/0.log"
Jul 29 22:22:44.295793 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:44.295765    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mmmvr_b6dc41d9-5eaa-482c-822a-b9e18c2f6b04/extractor/0.log"
Jul 29 22:22:45.650461 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:45.650429    2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jqkkn/perf-node-gather-daemonset-vwq6p"
Jul 29 22:22:46.755751 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:46.755714    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-mpcf4_38f5c933-cd6a-40ef-9a57-5d378f513282/server/0.log"
Jul 29 22:22:46.846908 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:46.846876    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pffpc_58cb8084-7ecc-40ab-a467-ca14d2e67719/manager/0.log"
Jul 29 22:22:46.970484 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:46.970429    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b55745745-gwjsn_8c7f984e-894a-4f5b-9419-b16627818310/manager/0.log"
Jul 29 22:22:46.991140 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:46.991111    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b55745745-gwjsn_8c7f984e-894a-4f5b-9419-b16627818310/copy-manifests/0.log"
Jul 29 22:22:48.871232 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:48.871204    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787cd99557-fzfmn_dbb736d7-8d40-4c69-990b-a3db196259aa/kube-auth-proxy/0.log"
Jul 29 22:22:50.520266 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:50.520238    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-787cd99557-fzfmn_dbb736d7-8d40-4c69-990b-a3db196259aa/kube-auth-proxy/0.log"
Jul 29 22:22:54.076396 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:54.076359    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl_1deada3b-1099-4beb-b51f-b4839686a1a8/kube-storage-version-migrator-operator/1.log"
Jul 29 22:22:54.077421 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:54.077396    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-gvgzl_1deada3b-1099-4beb-b51f-b4839686a1a8/kube-storage-version-migrator-operator/0.log"
Jul 29 22:22:55.175442 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.175414    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/kube-multus-additional-cni-plugins/0.log"
Jul 29 22:22:55.202304 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.202275    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/egress-router-binary-copy/0.log"
Jul 29 22:22:55.225620 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.225591    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/cni-plugins/0.log"
Jul 29 22:22:55.248478 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.248452    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/bond-cni-plugin/0.log"
Jul 29 22:22:55.270782 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.270751    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/routeoverride-cni/0.log"
Jul 29 22:22:55.294956 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.294924    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/whereabouts-cni-bincopy/0.log"
Jul 29 22:22:55.322347 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.322318    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zgsh_ca902c68-ff65-4b9b-8b08-dcb3e36e1d97/whereabouts-cni/0.log"
Jul 29 22:22:55.819301 ip-10-0-129-242 kubenswrapper[2572]: I0729 22:22:55.819268    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fvq2g_87efc84b-d7a4-489a-a301-4af34121c04b/kube-multus/0.log"