Jul 26 17:57:25.716493 ip-10-0-133-34 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 17:57:26.028929 ip-10-0-133-34 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 17:57:26.028929 ip-10-0-133-34 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 17:57:26.028929 ip-10-0-133-34 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 17:57:26.028929 ip-10-0-133-34 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 17:57:26.028929 ip-10-0-133-34 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 17:57:26.029763 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.029630    2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 17:57:26.033772 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033756    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:57:26.033772 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033771    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033776    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033780    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033784    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033787    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033790    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033792    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033795    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033798    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033800    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033803    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033806    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033808    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033811    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033813    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033816    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033819    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033822    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033824    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033827    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:57:26.033838 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033830    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033832    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033835    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033837    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033840    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033843    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033846    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033849    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033852    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033855    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033857    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033861    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033864    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033867    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033870    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033872    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033875    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033877    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033880    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033882    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:57:26.034306 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033885    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033887    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033890    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033892    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033894    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033897    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033899    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033902    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033904    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033907    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033909    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033913    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033918    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033921    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033925    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033928    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033930    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033933    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033936    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:57:26.034812 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033938    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033941    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033944    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033953    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033956    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033959    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033961    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033964    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033967    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033970    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033973    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033975    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033978    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033981    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033983    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033986    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033988    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033991    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033994    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.033997    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:57:26.035322 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034000    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034003    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034005    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034008    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034011    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034014    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034017    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034021    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034025    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034028    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034031    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034034    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034037    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034040    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034044    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034048    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034051    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034054    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:57:26.035819 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.034057    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036081    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036090    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036094    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036098    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036101    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036104    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036107    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036110    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036113    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036116    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036118    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036121    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036124    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036126    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036129    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036131    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036134    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036136    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036139    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:57:26.036253 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036141    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036144    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036146    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036149    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036153    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036157    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036161    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036165    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036169    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036172    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036174    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036177    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036180    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036183    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036186    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036189    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036193    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036195    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036198    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:57:26.036760 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036201    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036203    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036206    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036209    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036211    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036214    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036216    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036219    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036222    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036224    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036226    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036229    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036231    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036234    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036236    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036239    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036242    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036244    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036247    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036249    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:57:26.037237 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036251    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036254    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036256    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036260    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036263    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036266    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036268    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036272    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036275    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036278    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036281    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036284    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036286    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036289    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036291    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036294    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036296    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036299    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036301    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036303    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:57:26.037938 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036306    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036308    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036311    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036313    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036316    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036318    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036321    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036323    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036327    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036330    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036332    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036336    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036339    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036343    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036346    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036349    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036352    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036355    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036357    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:57:26.038632 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036361    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036364    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036789    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036796    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036799    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036802    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036806    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036809    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036811    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036815    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036819    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036823    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036826    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036828    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036831    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036833    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036836    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036839    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036842    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036844    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:57:26.039197 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036847    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036850    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036852    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036855    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036857    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036860    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036863    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036865    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036867    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036870    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036872    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036875    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036878    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036882    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036886    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036889    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036892    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036895    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036897    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:57:26.039707 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036900    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036904    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036906    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036909    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036912    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036916    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036919    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036921    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036924    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036927    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036929    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036932    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036935    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036937    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036940    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036943    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036945    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036947    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036950    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:57:26.040184 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036953    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036955    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036958    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036960    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036963    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036965    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036968    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036971    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036973    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036976    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036979    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036982    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036985    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036988    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036990    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036993    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036995    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.036998    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037000    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037003    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:57:26.040680 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037005    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037008    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037010    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037012    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037015    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037017    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037020    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037022    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037025    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037027    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037030    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037032    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037035    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037038    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037040    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037043    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037045    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037048    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037050    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037053    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:57:26.041196 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037056    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037058    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.037061    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037125    2565 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037132    2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037139    2565 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037143    2565 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037147    2565 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037150    2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037155    2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037159    2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037162    2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037165    2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037168    2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037172    2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037175    2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037178    2565 flags.go:64] FLAG: --cgroup-root=""
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037181    2565 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037184    2565 flags.go:64] FLAG: --client-ca-file=""
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037187    2565 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037189    2565 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037194    2565 flags.go:64] FLAG: --cluster-domain=""
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037197    2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037200    2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 17:57:26.041706 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037203    2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037207    2565 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037210    2565 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037213    2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037216    2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037220    2565 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037225    2565 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037229    2565 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037232    2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037235    2565 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037238    2565 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037243    2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037246    2565 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037250    2565 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037253    2565 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037257    2565 flags.go:64] FLAG: --enable-server="true"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037259    2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037286    2565 flags.go:64] FLAG: --event-burst="100"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037289    2565 flags.go:64] FLAG: --event-qps="50"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037292    2565 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037296    2565 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037298    2565 flags.go:64] FLAG: --eviction-hard=""
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037310    2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037313    2565 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037316    2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 17:57:26.042282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037319    2565 flags.go:64] FLAG: --eviction-soft=""
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037323    2565 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037325    2565 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037329    2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037332    2565 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037335    2565 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037338    2565 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037341    2565 flags.go:64] FLAG: --feature-gates=""
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037345    2565 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037348    2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037351    2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037354    2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037357    2565 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037362    2565 flags.go:64] FLAG: --help="false"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037365    2565 flags.go:64] FLAG: --hostname-override="ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037368    2565 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037371    2565 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037374    2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037378    2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037381    2565 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037384    2565 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037441    2565 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037495    2565 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 17:57:26.042900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037499    2565 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037503    2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037506    2565 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037510    2565 flags.go:64] FLAG: --kube-reserved=""
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037513    2565 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037516    2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037520    2565 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037523    2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037530    2565 flags.go:64] FLAG: --lock-file=""
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037533    2565 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037536    2565 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037540    2565 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037547    2565 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037550    2565 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037554    2565 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037557    2565 flags.go:64] FLAG: --logging-format="text"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.037567    2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039032    2565 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039167    2565 flags.go:64] FLAG: --manifest-url=""
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039175    2565 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039191    2565 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039195    2565 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039199    2565 flags.go:64] FLAG: --max-pods="110"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039203    2565 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039207    2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 17:57:26.043497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039210    2565 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039213    2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039217    2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039221    2565 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039224    2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039233    2565 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039237    2565 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039240    2565 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039243    2565 flags.go:64] FLAG: --pod-cidr=""
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039246    2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039252    2565 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039255    2565 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039258    2565 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039261    2565 flags.go:64] FLAG: --port="10250"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039264    2565 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039268    2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0277aff3b88b9798e"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039271    2565 flags.go:64] FLAG: --qos-reserved=""
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039274    2565 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039277    2565 flags.go:64] FLAG: --register-node="true"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039280    2565 flags.go:64] FLAG: --register-with-taints=""
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039284    2565 flags.go:64] FLAG: --registry-burst="10"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039287    2565 flags.go:64] FLAG: --registry-qps="5"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039290    2565 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039293    2565 flags.go:64] FLAG: --reserved-memory=""
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039297    2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 17:57:26.044151 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039301    2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039313    2565 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039317    2565 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039320    2565 flags.go:64] FLAG: --runonce="false"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039323    2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039326    2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039329    2565 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039332    2565 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039335    2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039338    2565 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039341    2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039344    2565 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039347    2565 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039350    2565 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039353    2565 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039356    2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039359    2565 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039362    2565 flags.go:64] FLAG: --system-cgroups=""
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039365    2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039370    2565 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039373    2565 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039376    2565 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039385    2565 flags.go:64] FLAG: --tls-min-version=""
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039388    2565 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039391    2565 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 17:57:26.044764 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039393    2565 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039396    2565 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039400    2565 flags.go:64] FLAG: --v="2"
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039404    2565 flags.go:64] FLAG: --version="false"
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039408    2565 flags.go:64] FLAG: --vmodule=""
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039411    2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.039415    2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039541    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039545    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039548    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039551    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039554    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039556    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039559    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039562    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039565    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039567    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039570    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039572    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039575    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039577    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:57:26.045383 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039580    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039583    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039585    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039588    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039591    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039593    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039596    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039598    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039601    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039604    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039606    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039609    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039612    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039615    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039617    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039620    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039622    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039626    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039631    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:57:26.045939 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039635    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039639    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039642    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039644    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039647    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039650    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039653    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039656    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039658    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039661    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039664    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039666    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039669    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039671    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039674    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039677    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039679    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039682    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039684    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039687    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:57:26.046409 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039689    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039692    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039708    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039712    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039715    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039718    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039721    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039724    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039727    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039729    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039732    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039736    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039741    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039744    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039747    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039749    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039752    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039755    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039757    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039760    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:57:26.046907 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039762    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039765    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039768    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039770    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039773    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039775    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039778    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039781    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039783    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039786    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039789    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039791    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039794    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039797    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039799    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039802    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039804    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039807    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039810    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039812    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:57:26.047402 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039816    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039820    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039824    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039826    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039829    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.039832    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.040387    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047068    2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047084    2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047105    2565 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047111    2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047167    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047171    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047175    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047178    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:57:26.047905 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047181    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047184    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047187    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047189    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047192    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047195    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047197    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047200    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047203    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047206    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047208    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047211    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047214    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047218    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047221    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047224    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047227    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047229    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047233    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:57:26.048271 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047237    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047241    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047244    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047247    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047249    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047252    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047256    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047259    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047262    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047264    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047267    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047270    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047273    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047276    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047279    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047281    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047284    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047287    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047289    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047292    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:57:26.048744 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047294    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047297    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047299    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047308    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047312    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047314    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047317    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047320    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047322    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047325    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047327    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047330    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047333    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047335    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047338    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047340    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047343    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047347    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047351    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:57:26.049242 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047354    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047356    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047360    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047362    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047365    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047367    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047370    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047372    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047375    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047377    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047380    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047382    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047385    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047387    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047390    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047392    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047395    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047398    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047401    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047403    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:57:26.049713 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047406    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047409    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047412    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047414    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047417    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047419    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047422    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047425    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047428    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047431    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047433    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047435    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047438    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047441    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047444    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047447    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:57:26.050205 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047449    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047455    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047578    2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047583    2565 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047586    2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047589    2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047592    2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047595    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047598    2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047600    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047604    2565 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047607    2565 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047610    2565 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047613    2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047615    2565 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 17:57:26.050659 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047618    2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047622    2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047626    2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047629    2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047632    2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047635    2565 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047638    2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047640    2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047643    2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047646    2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047648    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047651    2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047653    2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047657    2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047660    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047663    2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047666    2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047668    2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047670    2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 17:57:26.051097 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047673    2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047675    2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047678    2565 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047680    2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047683    2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047685    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047688    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047690    2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047693    2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047708    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047712    2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047715    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047718    2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047720    2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047723    2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047726    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047728    2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047731    2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047734    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047736    2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 17:57:26.051594 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047740    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047742    2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047745    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047749    2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047752    2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047755    2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047758    2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047760    2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047763    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047773    2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047776    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047779    2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047781    2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047784    2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047787    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047790    2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047792    2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047794    2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047797    2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 17:57:26.052164 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047799    2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047802    2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047805    2565 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047808    2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047810    2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047813    2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047815    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047818    2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047820    2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047823    2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047826    2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047829    2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047831    2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047834    2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047837    2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047839    2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047842    2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047844    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047847    2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047849    2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 17:57:26.052837 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047852    2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047854    2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047857    2565 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047860    2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047863    2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047865    2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047868    2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:26.047870    2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047876    2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.047993    2565 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.050302    2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.051468    2565 server.go:1019] "Starting client certificate rotation"
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.051590    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 17:57:26.053358 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.051643    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 17:57:26.053882 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.053675    2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 17:57:26.055384 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.055359    2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 17:57:26.063816 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.063794    2565 log.go:25] "Validated CRI v1 runtime API"
Jul 26 17:57:26.065671 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.065655    2565 log.go:25] "Validated CRI v1 image API"
Jul 26 17:57:26.066522 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.066504    2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 17:57:26.069110 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.069090    2565 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 83b5f0f6-fa6a-4ca1-82d5-07651b8842e3:/dev/nvme0n1p3 9e0f748b-04a2-419e-9dac-a3da874192fb:/dev/nvme0n1p4]
Jul 26 17:57:26.069188 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.069109    2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 17:57:26.075762 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.075615    2565 manager.go:216] Machine: {Timestamp:2026-07-26 17:57:26.074507895 +0000 UTC m=+0.264537641 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104299 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24641816d8b7951e4ccc8f579a106b SystemUUID:ec246418-16d8-b795-1e4c-cc8f579a106b BootID:4576db21-c9ab-4695-92d4-9726bd07579d Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:47:09:95:20:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:47:09:95:20:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:8f:0e:26:12:c7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 17:57:26.075762 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.075748    2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 17:57:26.075904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.075835    2565 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 17:57:26.077279 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.077261    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 17:57:26.078176 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.078141    2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 17:57:26.078347 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.078184    2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-34.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 17:57:26.079159 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.079146    2565 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 17:57:26.079213 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.079166    2565 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 17:57:26.079213 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.079179    2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 17:57:26.079732 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.079720    2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 17:57:26.080944 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.080932    2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 17:57:26.084736 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084722    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 17:57:26.084877 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084866    2565 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 17:57:26.084940 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084908    2565 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 17:57:26.084940 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084924    2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 17:57:26.085025 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084946    2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 17:57:26.085025 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084959    2565 kubelet.go:398] "Adding apiserver pod source"
Jul 26 17:57:26.085025 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.084973    2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 17:57:26.085352 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.085338    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 17:57:26.085413 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.085361    2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 17:57:26.087144 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.087130    2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 17:57:26.088499 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.088486    2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 17:57:26.088544 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.088516    2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089172    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089187    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089195    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089201    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089210    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089216    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089225    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089232    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089239    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089245    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089255    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 17:57:26.089414 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089280    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 17:57:26.089888 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089878    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 17:57:26.089888 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.089887    2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 17:57:26.093325 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.093309    2565 server.go:1268] "Started kubelet"
Jul 26 17:57:26.093521 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.093466    2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 17:57:26.093629 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.093547    2565 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 17:57:26.093913 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.093882    2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 17:57:26.094262 ip-10-0-133-34 systemd[1]: Started Kubernetes Kubelet.
Jul 26 17:57:26.094882 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.094687    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hlc4"
Jul 26 17:57:26.095483 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.095300    2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 17:57:26.095483 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.095326    2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-34.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 17:57:26.095615 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.095485    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 17:57:26.095664 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.095616    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 17:57:26.095752 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.095726    2565 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 17:57:26.099747 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.099722    2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 17:57:26.103140 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.103121    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4hlc4"
Jul 26 17:57:26.103217 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.103211    2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 17:57:26.103310 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.103292    2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 17:57:26.103423 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.103403    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.104130 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.103941    2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 17:57:26.104130 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.104061    2565 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 17:57:26.104251 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.104142    2565 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 17:57:26.104438 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.104420    2565 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 17:57:26.104438 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.104439    2565 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 17:57:26.104946 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.104926    2565 factory.go:55] Registering systemd factory
Jul 26 17:57:26.105023 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.104980    2565 factory.go:223] Registration of the systemd container factory successfully
Jul 26 17:57:26.106245 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.105290    2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-34.ec2.internal.18c5e8aa72f919ec  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-34.ec2.internal,UID:ip-10-0-133-34.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-34.ec2.internal,},FirstTimestamp:2026-07-26 17:57:26.093281772 +0000 UTC m=+0.283311520,LastTimestamp:2026-07-26 17:57:26.093281772 +0000 UTC m=+0.283311520,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-34.ec2.internal,}"
Jul 26 17:57:26.106569 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.106552    2565 factory.go:153] Registering CRI-O factory
Jul 26 17:57:26.106691 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.106680    2565 factory.go:223] Registration of the crio container factory successfully
Jul 26 17:57:26.106837 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.106826    2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 17:57:26.106938 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.106929    2565 factory.go:103] Registering Raw factory
Jul 26 17:57:26.107015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.107008    2565 manager.go:1196] Started watching for new ooms in manager
Jul 26 17:57:26.107914 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.107894    2565 manager.go:349] Starting recovery of all containers
Jul 26 17:57:26.108211 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.108181    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 17:57:26.108339 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.108314    2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-34.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 17:57:26.114007 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.113979    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 17:57:26.119513 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.119494    2565 manager.go:354] Recovery completed
Jul 26 17:57:26.123896 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.123831    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.127866 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.127846    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.127935 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.127877    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.127935 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.127889    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.128437 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.128419    2565 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 17:57:26.128437 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.128434    2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 17:57:26.128583 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.128435    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.128583 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.128454    2565 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 17:57:26.129246 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.129230    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.129311 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.129262    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.129311 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.129278    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.130686 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.130672    2565 policy_none.go:49] "None policy: Start"
Jul 26 17:57:26.130745 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.130689    2565 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 17:57:26.130745 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.130700    2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 17:57:26.132882 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.132867    2565 policy_none.go:47] "Start"
Jul 26 17:57:26.182656 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.182440    2565 manager.go:337] "Starting Device Plugin manager"
Jul 26 17:57:26.182788 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.182676    2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 17:57:26.182788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.182690    2565 server.go:87] "Starting device plugin registration server"
Jul 26 17:57:26.183033 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.183016    2565 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 17:57:26.183075 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.183034    2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 17:57:26.183159 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.183130    2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 17:57:26.183275 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.183238    2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 17:57:26.183275 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.183252    2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 17:57:26.183917 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.183896    2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 17:57:26.184010 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.183957    2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.283640 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.283558    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.285964 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.284675    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.285964 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.284707    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.285964 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.284738    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.285964 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.284773    2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.290832 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.290811    2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.290898 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.290841    2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-34.ec2.internal\": node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.310658 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.310634    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.312544 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.312516    2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 17:57:26.312671 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.312561    2565 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 17:57:26.312671 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.312587    2565 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 17:57:26.312671 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.312630    2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 17:57:26.315230 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.315205    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:57:26.411727 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.411663    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.412855 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.412830    2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"]
Jul 26 17:57:26.412954 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.412941    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.414675 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.414656    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.414789 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.414688    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.414789 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.414699    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.417160 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417147    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.417282 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417265    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.417323 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417298    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.417952 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417935    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.418045 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417969    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.418045 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417985    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.418045 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.417935    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.418045 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.418043    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.418234 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.418053    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.420500 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.420482    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.420555 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.420512    2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 17:57:26.421233 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.421219    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 17:57:26.421320 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.421244    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 17:57:26.421320 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.421261    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeHasSufficientPID"
Jul 26 17:57:26.444604 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.444575    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-34.ec2.internal\" not found" node="ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.449510 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.449493    2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-34.ec2.internal\" not found" node="ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.506829 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.506794    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15f7650d29b3cd0a467b00af3cc82a94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal\" (UID: \"15f7650d29b3cd0a467b00af3cc82a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.506829 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.506831    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15f7650d29b3cd0a467b00af3cc82a94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal\" (UID: \"15f7650d29b3cd0a467b00af3cc82a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.507034 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.506852    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd7040ec0042cf6f2efb0378a077f8ac-config\") pod \"kube-apiserver-proxy-ip-10-0-133-34.ec2.internal\" (UID: \"bd7040ec0042cf6f2efb0378a077f8ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.511906 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.511876    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.607138 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.607049    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15f7650d29b3cd0a467b00af3cc82a94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal\" (UID: \"15f7650d29b3cd0a467b00af3cc82a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.607138 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.607093    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15f7650d29b3cd0a467b00af3cc82a94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal\" (UID: \"15f7650d29b3cd0a467b00af3cc82a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.607138 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.607114    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd7040ec0042cf6f2efb0378a077f8ac-config\") pod \"kube-apiserver-proxy-ip-10-0-133-34.ec2.internal\" (UID: \"bd7040ec0042cf6f2efb0378a077f8ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.607345 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.607145    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15f7650d29b3cd0a467b00af3cc82a94-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal\" (UID: \"15f7650d29b3cd0a467b00af3cc82a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.607345 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.607148    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15f7650d29b3cd0a467b00af3cc82a94-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal\" (UID: \"15f7650d29b3cd0a467b00af3cc82a94\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.607345 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.607166    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd7040ec0042cf6f2efb0378a077f8ac-config\") pod \"kube-apiserver-proxy-ip-10-0-133-34.ec2.internal\" (UID: \"bd7040ec0042cf6f2efb0378a077f8ac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.612223 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.612193    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.712830 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.712794    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.747743 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.747692    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.753546 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:26.753327    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:26.813669 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.813633    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:26.914270 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:26.914193    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:27.014467 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:27.014422    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:27.051203 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.051176    2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 17:57:27.051667 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.051333    2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 17:57:27.109833 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.109787    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 17:52:27 +0000 UTC" deadline="2027-12-30 05:09:07.760542076 +0000 UTC"
Jul 26 17:57:27.109989 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.109821    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12515h11m40.650724534s"
Jul 26 17:57:27.115445 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:27.115425    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:27.129557 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.129535    2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:57:27.215858 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:27.215826    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:27.316786 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:27.316747    2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-34.ec2.internal\" not found"
Jul 26 17:57:27.330485 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:27.330452    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f7650d29b3cd0a467b00af3cc82a94.slice/crio-80aa1c92a284d14d8ad1581ec6bf54ad5680d781c721b3c0db4104e6c192837b WatchSource:0}: Error finding container 80aa1c92a284d14d8ad1581ec6bf54ad5680d781c721b3c0db4104e6c192837b: Status 404 returned error can't find the container with id 80aa1c92a284d14d8ad1581ec6bf54ad5680d781c721b3c0db4104e6c192837b
Jul 26 17:57:27.330936 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:27.330921    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7040ec0042cf6f2efb0378a077f8ac.slice/crio-a18001878bb4f2b94f3d0466c827f16add34db1914000f89d03bd6d81b727b23 WatchSource:0}: Error finding container a18001878bb4f2b94f3d0466c827f16add34db1914000f89d03bd6d81b727b23: Status 404 returned error can't find the container with id a18001878bb4f2b94f3d0466c827f16add34db1914000f89d03bd6d81b727b23
Jul 26 17:57:27.336729 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.336710    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 17:57:27.347314 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.347292    2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:57:27.403928 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.403885    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:27.416037 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.416008    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 17:57:27.416977 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.416963    2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal"
Jul 26 17:57:27.427782 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.427760    2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 17:57:27.537189 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.537116    2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:57:27.898483 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:27.898267    2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 17:57:28.086967 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.086940    2565 apiserver.go:52] "Watching apiserver"
Jul 26 17:57:28.093522 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.093476    2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 17:57:28.094930 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.094900    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-cvvsr","kube-system/global-pull-secret-syncer-4k55t","kube-system/konnectivity-agent-fkfch","openshift-cluster-node-tuning-operator/tuned-dtpb6","openshift-dns/node-resolver-7jpg7","openshift-multus/multus-additional-cni-plugins-ljtcc","openshift-multus/multus-rtr8b","openshift-multus/network-metrics-daemon-nthlf","openshift-network-diagnostics/network-check-target-n68jr","kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954","openshift-image-registry/node-ca-pqvrw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal","openshift-network-operator/iptables-alerter-tfpnp"]
Jul 26 17:57:28.097731 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.097697    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.100635 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.100609    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 17:57:28.101597 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.100749    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kfj29\""
Jul 26 17:57:28.101597 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.100930    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.101597 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.101047    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.101597 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.101554    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.103180 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.103159    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 17:57:28.105563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.104715    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.105563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.104906    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.105563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.104956    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 17:57:28.105563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.105161    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.105563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.105371    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zlp54\""
Jul 26 17:57:28.105563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.105527    2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 17:57:28.106844 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.106825    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.109096 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.108650    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 17:57:28.109096 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.108932    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 17:57:28.109312 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.109194    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4krpb\""
Jul 26 17:57:28.110042 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.109762    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.110412 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.110372    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.110502 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.110416    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.110735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.110714    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xb9tf\""
Jul 26 17:57:28.113549 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.112887    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.113549 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.113374    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.113549 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.113407    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.113549 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.113415    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rr5cc\""
Jul 26 17:57:28.115859 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.115838    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lrrps\""
Jul 26 17:57:28.115989 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.115899    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 17:57:28.116149 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.116118    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 17:57:28.116273 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.116253    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:28.116409 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.116341    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:28.117757 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.117681    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-hostroot\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.117883    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-modprobe-d\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.117922    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysconfig\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.117946    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-kubernetes\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.117972    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-var-lib-kubelet\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.117997    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-host\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118028    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbzc2\" (UniqueName: \"kubernetes.io/projected/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-kube-api-access-sbzc2\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118062    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-system-cni-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118086    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-cni-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3562f0e-f4c2-447d-a176-0596c7b205bd-cni-binary-copy\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118133    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-netns\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118156    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-kubelet\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118182    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-multus-certs\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118206    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sbpg\" (UniqueName: \"kubernetes.io/projected/e3562f0e-f4c2-447d-a176-0596c7b205bd-kube-api-access-6sbpg\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118230    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-daemon-config\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118253    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-sys\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118277    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-tuned\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.118934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118317    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-tmp\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118342    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-etc-kubernetes\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118365    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysctl-conf\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118387    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-cnibin\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-socket-dir-parent\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118445    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-cni-multus\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118470    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a50ffe58-a86d-46f0-9feb-5d96623d19db-agent-certs\") pod \"konnectivity-agent-fkfch\" (UID: \"a50ffe58-a86d-46f0-9feb-5d96623d19db\") " pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-os-release\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118577    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-cni-bin\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118603    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-device-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118626    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-etc-selinux\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118666    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-sys-fs\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118687    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-conf-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118718    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118761    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-socket-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118799    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysctl-d\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118822    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-systemd\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.119904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118849    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-k8s-cni-cncf-io\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.120872 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-registration-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.120872 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118931    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a50ffe58-a86d-46f0-9feb-5d96623d19db-konnectivity-ca\") pod \"konnectivity-agent-fkfch\" (UID: \"a50ffe58-a86d-46f0-9feb-5d96623d19db\") " pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.120872 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.118983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-run\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.120872 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.119008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-lib-modules\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.120872 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.119053    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfc79\" (UniqueName: \"kubernetes.io/projected/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-kube-api-access-nfc79\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.120872 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.120847    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.121178 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.120915    2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 17:57:28.123880 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.123858    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hsqm4\""
Jul 26 17:57:28.124035 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.123934    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.124103 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.123861    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.124177 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.124161    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 17:57:28.124357 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.124338    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.124448 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.124425    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:28.126964 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.126927    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.127065 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.127031    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:28.129851 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.129601    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.132628 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.132267    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.133674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133040    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.133674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133092    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xmhqn\""
Jul 26 17:57:28.133674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133264    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 17:57:28.133674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133360    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 17:57:28.133674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133454    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.133674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133535    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 17:57:28.133999 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.133759    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 17:57:28.135451 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.135250    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 17:57:28.135451 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.135270    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 17:57:28.135451 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.135391    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 17:57:28.135660 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.135510    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n798h\""
Jul 26 17:57:28.138435 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.138413    2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8xjjt"
Jul 26 17:57:28.146051 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.146029    2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8xjjt"
Jul 26 17:57:28.205200 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.205167    2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 17:57:28.220191 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220153    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220209    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-registration-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220229    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a50ffe58-a86d-46f0-9feb-5d96623d19db-konnectivity-ca\") pod \"konnectivity-agent-fkfch\" (UID: \"a50ffe58-a86d-46f0-9feb-5d96623d19db\") " pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220246    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-run\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220269    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-lib-modules\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220294    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-hostroot\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220341    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-registration-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.220360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220355    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysconfig\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220392    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-kubernetes\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220410    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysconfig\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220421    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-host\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220445    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-system-cni-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220468    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-cni-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220491    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-kubelet\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220515    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-multus-certs\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220544    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-cni-bin\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220570    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-ovnkube-config\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220595    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-env-overrides\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/66f1b080-76d2-4286-912b-cb3d61403c70-ovn-node-metrics-cert\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-daemon-config\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220668    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-systemd-units\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220691    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-ovnkube-script-lib\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.220735 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220716    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-system-cni-dir\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220764    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-cnibin\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220767    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-cni-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220790    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-etc-kubernetes\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220839    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-kubernetes\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220849    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46xk\" (UniqueName: \"kubernetes.io/projected/17b413f2-3a11-4e6e-bd9e-2c72c576749d-kube-api-access-f46xk\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220877    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-cni-multus\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220912    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-system-cni-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220931    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-var-lib-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-ovn\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220986    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-os-release\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221013    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-cni-bin\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221040    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b6e1a55-1afc-40a7-be4c-9f5810802717-kubelet-config\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221066    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-slash\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221093    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-cni-netd\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.221534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221119    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-sys-fs\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221153    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-socket-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221178    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-systemd\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221207    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt49q\" (UniqueName: \"kubernetes.io/projected/85696f9e-3458-467e-bf4b-9503978d99ca-kube-api-access-rt49q\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtsxw\" (UniqueName: \"kubernetes.io/projected/66f1b080-76d2-4286-912b-cb3d61403c70-kube-api-access-mtsxw\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221259    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-k8s-cni-cncf-io\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221282    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfc79\" (UniqueName: \"kubernetes.io/projected/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-kube-api-access-nfc79\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221306    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb7571f6-7ed7-4b3e-b331-b0855c53f528-iptables-alerter-script\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221330    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-lib-modules\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221334    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j45h4\" (UniqueName: \"kubernetes.io/projected/caba9223-8c40-4e7e-9740-98fcb97755b0-kube-api-access-j45h4\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221360    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/85696f9e-3458-467e-bf4b-9503978d99ca-serviceca\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-hostroot\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b6e1a55-1afc-40a7-be4c-9f5810802717-dbus\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221432    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-modprobe-d\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221457    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-var-lib-kubelet\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221487    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvf4z\" (UniqueName: \"kubernetes.io/projected/cb7571f6-7ed7-4b3e-b331-b0855c53f528-kube-api-access-jvf4z\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221512    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85696f9e-3458-467e-bf4b-9503978d99ca-host\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.222431 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221535    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17b413f2-3a11-4e6e-bd9e-2c72c576749d-hosts-file\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221559    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-systemd\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221583    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-etc-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221608    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-os-release\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221640    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbzc2\" (UniqueName: \"kubernetes.io/projected/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-kube-api-access-sbzc2\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221668    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3562f0e-f4c2-447d-a176-0596c7b205bd-cni-binary-copy\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221672    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-kubelet\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221695    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-netns\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221711    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-multus-certs\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221722    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-etc-kubernetes\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221724    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sbpg\" (UniqueName: \"kubernetes.io/projected/e3562f0e-f4c2-447d-a176-0596c7b205bd-kube-api-access-6sbpg\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-tuned\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-tmp\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-os-release\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.220881    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-host\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-socket-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221924    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-cni-multus\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221154    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a50ffe58-a86d-46f0-9feb-5d96623d19db-konnectivity-ca\") pod \"konnectivity-agent-fkfch\" (UID: \"a50ffe58-a86d-46f0-9feb-5d96623d19db\") " pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.223301 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.221208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-run\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-kubelet\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222077    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-systemd\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222115    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-k8s-cni-cncf-io\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222146    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-sys\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222150    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-modprobe-d\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222176    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222203    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-var-lib-kubelet\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222207    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs74d\" (UniqueName: \"kubernetes.io/projected/06937257-8927-494e-a4a6-532585fdc8e8-kube-api-access-hs74d\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222217    2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222238    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysctl-conf\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222313    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-log-socket\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222340    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222369    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-cnibin\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222398    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-socket-dir-parent\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222425    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb7571f6-7ed7-4b3e-b331-b0855c53f528-host-slash\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222452    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.224206 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222480    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222537    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a50ffe58-a86d-46f0-9feb-5d96623d19db-agent-certs\") pod \"konnectivity-agent-fkfch\" (UID: \"a50ffe58-a86d-46f0-9feb-5d96623d19db\") " pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222598    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-device-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222629    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-etc-selinux\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222660    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222690    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222720    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-conf-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222748    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222758    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-daemon-config\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysctl-d\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222891    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17b413f2-3a11-4e6e-bd9e-2c72c576749d-tmp-dir\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222918    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-run-netns\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.222943    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-node-log\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223272    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-sys-fs\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-run-netns\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223353    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-host-var-lib-cni-bin\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223437    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysctl-conf\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223545    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-sysctl-d\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223591    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-socket-dir-parent\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.223635    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-cnibin\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.224016    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-device-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.224114    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3562f0e-f4c2-447d-a176-0596c7b205bd-cni-binary-copy\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.224118    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-etc-selinux\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.224167    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-sys\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.224182    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3562f0e-f4c2-447d-a176-0596c7b205bd-multus-conf-dir\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.225987 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.224244    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.226470 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.226267    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-etc-tuned\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.226555 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.226535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a50ffe58-a86d-46f0-9feb-5d96623d19db-agent-certs\") pod \"konnectivity-agent-fkfch\" (UID: \"a50ffe58-a86d-46f0-9feb-5d96623d19db\") " pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.226619 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.226590    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-tmp\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.236896 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.236871    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbzc2\" (UniqueName: \"kubernetes.io/projected/6a3ca5ad-c732-4b8c-bc71-ddbf59eff096-kube-api-access-sbzc2\") pod \"aws-ebs-csi-driver-node-gl954\" (UID: \"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.237590 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.237563    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sbpg\" (UniqueName: \"kubernetes.io/projected/e3562f0e-f4c2-447d-a176-0596c7b205bd-kube-api-access-6sbpg\") pod \"multus-rtr8b\" (UID: \"e3562f0e-f4c2-447d-a176-0596c7b205bd\") " pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.239279 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.239230    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfc79\" (UniqueName: \"kubernetes.io/projected/9f0c8ff6-aa29-4b61-aed1-7e1535304b49-kube-api-access-nfc79\") pod \"tuned-dtpb6\" (UID: \"9f0c8ff6-aa29-4b61-aed1-7e1535304b49\") " pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.318281 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.317929    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal" event={"ID":"15f7650d29b3cd0a467b00af3cc82a94","Type":"ContainerStarted","Data":"80aa1c92a284d14d8ad1581ec6bf54ad5680d781c721b3c0db4104e6c192837b"}
Jul 26 17:57:28.319971 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.319942    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal" event={"ID":"bd7040ec0042cf6f2efb0378a077f8ac","Type":"ContainerStarted","Data":"a18001878bb4f2b94f3d0466c827f16add34db1914000f89d03bd6d81b727b23"}
Jul 26 17:57:28.323221 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323185    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.323353 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323230    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:28.323353 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323260    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17b413f2-3a11-4e6e-bd9e-2c72c576749d-tmp-dir\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.323699 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323537    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-run-netns\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.323699 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323575    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17b413f2-3a11-4e6e-bd9e-2c72c576749d-tmp-dir\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.323699 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-node-log\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.323699 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323591    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-run-netns\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.323699 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323619    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.323699 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323682    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-cni-bin\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323711    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-node-log\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323709    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-ovnkube-config\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323783    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-env-overrides\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323807    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/66f1b080-76d2-4286-912b-cb3d61403c70-ovn-node-metrics-cert\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323826    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-systemd-units\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-ovnkube-script-lib\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323861    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-system-cni-dir\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323883    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323894    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-cnibin\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323917    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46xk\" (UniqueName: \"kubernetes.io/projected/17b413f2-3a11-4e6e-bd9e-2c72c576749d-kube-api-access-f46xk\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323952    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-var-lib-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-ovn\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324026    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b6e1a55-1afc-40a7-be4c-9f5810802717-kubelet-config\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324051    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-slash\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324074 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324076    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-cni-netd\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt49q\" (UniqueName: \"kubernetes.io/projected/85696f9e-3458-467e-bf4b-9503978d99ca-kube-api-access-rt49q\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324133    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtsxw\" (UniqueName: \"kubernetes.io/projected/66f1b080-76d2-4286-912b-cb3d61403c70-kube-api-access-mtsxw\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324135    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-cni-bin\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324162    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb7571f6-7ed7-4b3e-b331-b0855c53f528-iptables-alerter-script\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j45h4\" (UniqueName: \"kubernetes.io/projected/caba9223-8c40-4e7e-9740-98fcb97755b0-kube-api-access-j45h4\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324217    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/85696f9e-3458-467e-bf4b-9503978d99ca-serviceca\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b6e1a55-1afc-40a7-be4c-9f5810802717-dbus\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324273    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvf4z\" (UniqueName: \"kubernetes.io/projected/cb7571f6-7ed7-4b3e-b331-b0855c53f528-kube-api-access-jvf4z\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324297    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85696f9e-3458-467e-bf4b-9503978d99ca-host\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324321    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17b413f2-3a11-4e6e-bd9e-2c72c576749d-hosts-file\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324323    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-env-overrides\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324346    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-systemd\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324350    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-ovnkube-config\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324375    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-etc-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324388    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-slash\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324402    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-os-release\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.324942 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324432    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-cni-netd\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324435    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-kubelet\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324456    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-system-cni-dir\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-kubelet\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324530    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-cnibin\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.324572    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.324648    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:28.8246256 +0000 UTC m=+3.014655340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324683    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324957    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/85696f9e-3458-467e-bf4b-9503978d99ca-serviceca\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325014    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-run-ovn-kubernetes\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323952    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-systemd-units\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325078    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-var-lib-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325151    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-ovn\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325225    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3b6e1a55-1afc-40a7-be4c-9f5810802717-kubelet-config\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325250    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3b6e1a55-1afc-40a7-be4c-9f5810802717-dbus\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325283    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-etc-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325458    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85696f9e-3458-467e-bf4b-9503978d99ca-host\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.325948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325515    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/17b413f2-3a11-4e6e-bd9e-2c72c576749d-hosts-file\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325560    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/06937257-8927-494e-a4a6-532585fdc8e8-os-release\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325568    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-systemd\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325575    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cb7571f6-7ed7-4b3e-b331-b0855c53f528-iptables-alerter-script\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.324479    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325635    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs74d\" (UniqueName: \"kubernetes.io/projected/06937257-8927-494e-a4a6-532585fdc8e8-kube-api-access-hs74d\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325667    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-log-socket\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325693    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325722    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb7571f6-7ed7-4b3e-b331-b0855c53f528-host-slash\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325747    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325790    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.325811    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.326114    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/66f1b080-76d2-4286-912b-cb3d61403c70-ovnkube-script-lib\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.326235    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.323680    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.326488    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-log-socket\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.326788 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.326608    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:28.327547 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.326680    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:28.826661439 +0000 UTC m=+3.016691190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:28.327547 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.326747    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb7571f6-7ed7-4b3e-b331-b0855c53f528-host-slash\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.327547 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.326792    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/66f1b080-76d2-4286-912b-cb3d61403c70-run-openvswitch\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.327547 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.326964    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/06937257-8927-494e-a4a6-532585fdc8e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.327759 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.327597    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/66f1b080-76d2-4286-912b-cb3d61403c70-ovn-node-metrics-cert\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.335590 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.335568    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:57:28.335698 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.335598    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:57:28.335698 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.335618    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:28.335698 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.335659    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:57:28.835648345 +0000 UTC m=+3.025678079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:28.338484 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.338455    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt49q\" (UniqueName: \"kubernetes.io/projected/85696f9e-3458-467e-bf4b-9503978d99ca-kube-api-access-rt49q\") pod \"node-ca-pqvrw\" (UID: \"85696f9e-3458-467e-bf4b-9503978d99ca\") " pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.339710 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.339613    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtsxw\" (UniqueName: \"kubernetes.io/projected/66f1b080-76d2-4286-912b-cb3d61403c70-kube-api-access-mtsxw\") pod \"ovnkube-node-cvvsr\" (UID: \"66f1b080-76d2-4286-912b-cb3d61403c70\") " pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.340515 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.340066    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46xk\" (UniqueName: \"kubernetes.io/projected/17b413f2-3a11-4e6e-bd9e-2c72c576749d-kube-api-access-f46xk\") pod \"node-resolver-7jpg7\" (UID: \"17b413f2-3a11-4e6e-bd9e-2c72c576749d\") " pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.341637 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.341452    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvf4z\" (UniqueName: \"kubernetes.io/projected/cb7571f6-7ed7-4b3e-b331-b0855c53f528-kube-api-access-jvf4z\") pod \"iptables-alerter-tfpnp\" (UID: \"cb7571f6-7ed7-4b3e-b331-b0855c53f528\") " pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.342458 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.342107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j45h4\" (UniqueName: \"kubernetes.io/projected/caba9223-8c40-4e7e-9740-98fcb97755b0-kube-api-access-j45h4\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.343254 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.343205    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs74d\" (UniqueName: \"kubernetes.io/projected/06937257-8927-494e-a4a6-532585fdc8e8-kube-api-access-hs74d\") pod \"multus-additional-cni-plugins-ljtcc\" (UID: \"06937257-8927-494e-a4a6-532585fdc8e8\") " pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.415469 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.415382    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rtr8b"
Jul 26 17:57:28.424346 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.424319    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954"
Jul 26 17:57:28.434856 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.434831    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:28.441898 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.441837    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dtpb6"
Jul 26 17:57:28.449847 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.449827    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7jpg7"
Jul 26 17:57:28.458825 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.458802    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ljtcc"
Jul 26 17:57:28.474844 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.474815    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tfpnp"
Jul 26 17:57:28.482948 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.482923    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:28.492923 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.492900    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pqvrw"
Jul 26 17:57:28.830768 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.830660    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:28.830949 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.830782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:28.830949 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.830866    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:28.830949 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.830894    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:28.830949 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.830942    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:29.83092521 +0000 UTC m=+4.020954949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:28.831121 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.830960    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:29.830952274 +0000 UTC m=+4.020982011 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:28.932023 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:28.931980    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:28.932208 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.932166    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:57:28.932208 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.932193    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:57:28.932208 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.932207    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:28.932395 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:28.932278    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:57:29.932258784 +0000 UTC m=+4.122288521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:29.149741 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:29.149621    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 17:52:28 +0000 UTC" deadline="2028-04-18 23:46:21.846429904 +0000 UTC"
Jul 26 17:57:29.149741 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:29.149667    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15173h48m52.696765866s"
Jul 26 17:57:29.408460 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.408431    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7571f6_7ed7_4b3e_b331_b0855c53f528.slice/crio-fb14ee2b44f881108566119751ad3e7f6865141eaeb8126a5083c1c858e5dded WatchSource:0}: Error finding container fb14ee2b44f881108566119751ad3e7f6865141eaeb8126a5083c1c858e5dded: Status 404 returned error can't find the container with id fb14ee2b44f881108566119751ad3e7f6865141eaeb8126a5083c1c858e5dded
Jul 26 17:57:29.411373 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.411342    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f1b080_76d2_4286_912b_cb3d61403c70.slice/crio-a301855c0341efaaa23b72179330113393abddfce754f9cdd3a1ace9dd12e238 WatchSource:0}: Error finding container a301855c0341efaaa23b72179330113393abddfce754f9cdd3a1ace9dd12e238: Status 404 returned error can't find the container with id a301855c0341efaaa23b72179330113393abddfce754f9cdd3a1ace9dd12e238
Jul 26 17:57:29.413562 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.413534    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3ca5ad_c732_4b8c_bc71_ddbf59eff096.slice/crio-f0876f632a19eaa5c4e3ffcc084bd498427b3aacfbfe7efd1b28d18a10322297 WatchSource:0}: Error finding container f0876f632a19eaa5c4e3ffcc084bd498427b3aacfbfe7efd1b28d18a10322297: Status 404 returned error can't find the container with id f0876f632a19eaa5c4e3ffcc084bd498427b3aacfbfe7efd1b28d18a10322297
Jul 26 17:57:29.414664 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.414632    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50ffe58_a86d_46f0_9feb_5d96623d19db.slice/crio-79473cff8380a5236b8b7ea4f28b2a7625df83575fc5b6313ddbfd5bcc4936f9 WatchSource:0}: Error finding container 79473cff8380a5236b8b7ea4f28b2a7625df83575fc5b6313ddbfd5bcc4936f9: Status 404 returned error can't find the container with id 79473cff8380a5236b8b7ea4f28b2a7625df83575fc5b6313ddbfd5bcc4936f9
Jul 26 17:57:29.415357 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.415311    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b413f2_3a11_4e6e_bd9e_2c72c576749d.slice/crio-8c57e5caa8241b6d271b36b22bf454993d8be219bab6d9b21331921f2acc3695 WatchSource:0}: Error finding container 8c57e5caa8241b6d271b36b22bf454993d8be219bab6d9b21331921f2acc3695: Status 404 returned error can't find the container with id 8c57e5caa8241b6d271b36b22bf454993d8be219bab6d9b21331921f2acc3695
Jul 26 17:57:29.416761 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.416445    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0c8ff6_aa29_4b61_aed1_7e1535304b49.slice/crio-bef473bc9d4fdeacb9baa7b087f87b17f693cad85d0c35053c8ebf32301fe4ad WatchSource:0}: Error finding container bef473bc9d4fdeacb9baa7b087f87b17f693cad85d0c35053c8ebf32301fe4ad: Status 404 returned error can't find the container with id bef473bc9d4fdeacb9baa7b087f87b17f693cad85d0c35053c8ebf32301fe4ad
Jul 26 17:57:29.418021 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.417998    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85696f9e_3458_467e_bf4b_9503978d99ca.slice/crio-4439370afab5ad39039578a9518035d79cb6ce808e9c05b88bb5ba6ce1f948d3 WatchSource:0}: Error finding container 4439370afab5ad39039578a9518035d79cb6ce808e9c05b88bb5ba6ce1f948d3: Status 404 returned error can't find the container with id 4439370afab5ad39039578a9518035d79cb6ce808e9c05b88bb5ba6ce1f948d3
Jul 26 17:57:29.418418 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.418397    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3562f0e_f4c2_447d_a176_0596c7b205bd.slice/crio-7f42f0c1283bb04ace74f5b6b768ff0e18cec7f1074ac17116aeabe0e328f1fe WatchSource:0}: Error finding container 7f42f0c1283bb04ace74f5b6b768ff0e18cec7f1074ac17116aeabe0e328f1fe: Status 404 returned error can't find the container with id 7f42f0c1283bb04ace74f5b6b768ff0e18cec7f1074ac17116aeabe0e328f1fe
Jul 26 17:57:29.421210 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:57:29.420827    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06937257_8927_494e_a4a6_532585fdc8e8.slice/crio-852af3dfa75b7364056aa632b4187a3f2b5da6ed612def8b7a2eb791de4dd0c6 WatchSource:0}: Error finding container 852af3dfa75b7364056aa632b4187a3f2b5da6ed612def8b7a2eb791de4dd0c6: Status 404 returned error can't find the container with id 852af3dfa75b7364056aa632b4187a3f2b5da6ed612def8b7a2eb791de4dd0c6
Jul 26 17:57:29.840351 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:29.839650    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:29.840351 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:29.839703    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:29.840351 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.839843    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:29.840351 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.839918    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:31.839896743 +0000 UTC m=+6.029926479 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:29.840351 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.839845    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:29.840351 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.839992    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:31.839971823 +0000 UTC m=+6.030001577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:29.941313 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:29.940594    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:29.941313 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.940820    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:57:29.941313 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.940848    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:57:29.941313 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.940863    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:29.941313 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:29.940927    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:57:31.940904583 +0000 UTC m=+6.130934331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:30.150533 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.150430    2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 17:52:28 +0000 UTC" deadline="2028-02-08 19:20:09.57783161 +0000 UTC"
Jul 26 17:57:30.150533 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.150476    2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13489h22m39.427360847s"
Jul 26 17:57:30.313899 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.313867    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:30.314078 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:30.314012    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:30.314508 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.314485    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:30.314608 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:30.314594    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:30.317387 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.317169    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:30.317387 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:30.317289    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:30.334158 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.334126    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fkfch" event={"ID":"a50ffe58-a86d-46f0-9feb-5d96623d19db","Type":"ContainerStarted","Data":"79473cff8380a5236b8b7ea4f28b2a7625df83575fc5b6313ddbfd5bcc4936f9"}
Jul 26 17:57:30.340727 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.340634    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954" event={"ID":"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096","Type":"ContainerStarted","Data":"f0876f632a19eaa5c4e3ffcc084bd498427b3aacfbfe7efd1b28d18a10322297"}
Jul 26 17:57:30.356451 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.356390    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"a301855c0341efaaa23b72179330113393abddfce754f9cdd3a1ace9dd12e238"}
Jul 26 17:57:30.364858 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.364829    2565 generic.go:356] "Generic (PLEG): container finished" podID="15f7650d29b3cd0a467b00af3cc82a94" containerID="56b3cc3373c4a660f84dcd1fe65e0b3a154adca41c4337506af7f80c31e697c0" exitCode=0
Jul 26 17:57:30.364975 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.364915    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal" event={"ID":"15f7650d29b3cd0a467b00af3cc82a94","Type":"ContainerDied","Data":"56b3cc3373c4a660f84dcd1fe65e0b3a154adca41c4337506af7f80c31e697c0"}
Jul 26 17:57:30.383747 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.383174    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal" event={"ID":"bd7040ec0042cf6f2efb0378a077f8ac","Type":"ContainerStarted","Data":"1e0a530ff2bb79692b16bbff9dbc565bef8597107c553c4687dffb43852752cc"}
Jul 26 17:57:30.389365 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.389247    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rtr8b" event={"ID":"e3562f0e-f4c2-447d-a176-0596c7b205bd","Type":"ContainerStarted","Data":"7f42f0c1283bb04ace74f5b6b768ff0e18cec7f1074ac17116aeabe0e328f1fe"}
Jul 26 17:57:30.396499 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.396434    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dtpb6" event={"ID":"9f0c8ff6-aa29-4b61-aed1-7e1535304b49","Type":"ContainerStarted","Data":"bef473bc9d4fdeacb9baa7b087f87b17f693cad85d0c35053c8ebf32301fe4ad"}
Jul 26 17:57:30.402894 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.402594    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfpnp" event={"ID":"cb7571f6-7ed7-4b3e-b331-b0855c53f528","Type":"ContainerStarted","Data":"fb14ee2b44f881108566119751ad3e7f6865141eaeb8126a5083c1c858e5dded"}
Jul 26 17:57:30.418492 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.418408    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerStarted","Data":"852af3dfa75b7364056aa632b4187a3f2b5da6ed612def8b7a2eb791de4dd0c6"}
Jul 26 17:57:30.436997 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.436963    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pqvrw" event={"ID":"85696f9e-3458-467e-bf4b-9503978d99ca","Type":"ContainerStarted","Data":"4439370afab5ad39039578a9518035d79cb6ce808e9c05b88bb5ba6ce1f948d3"}
Jul 26 17:57:30.450684 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:30.450659    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7jpg7" event={"ID":"17b413f2-3a11-4e6e-bd9e-2c72c576749d","Type":"ContainerStarted","Data":"8c57e5caa8241b6d271b36b22bf454993d8be219bab6d9b21331921f2acc3695"}
Jul 26 17:57:31.472932 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:31.472846    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal" event={"ID":"15f7650d29b3cd0a467b00af3cc82a94","Type":"ContainerStarted","Data":"c8a41d46c63ee744efd89f55d97939342c2a81e1ecf99f31a2927526c6e072c1"}
Jul 26 17:57:31.489696 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:31.489634    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-34.ec2.internal" podStartSLOduration=3.489610798 podStartE2EDuration="3.489610798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:30.39961597 +0000 UTC m=+4.589645729" watchObservedRunningTime="2026-07-26 17:57:31.489610798 +0000 UTC m=+5.679640559"
Jul 26 17:57:31.860285 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:31.859405    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:31.860285 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:31.859465    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:31.860285 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.859626    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:31.860285 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.859691    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:35.859672451 +0000 UTC m=+10.049702189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:31.860285 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.860129    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:31.860285 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.860189    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:35.860171753 +0000 UTC m=+10.050201493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:31.960610 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:31.960569    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:31.960830 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.960776    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:57:31.960830 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.960802    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:57:31.960830 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.960816    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:31.961018 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:31.960884    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:57:35.960861787 +0000 UTC m=+10.150891525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:32.316562 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:32.315572    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:32.316562 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:32.315723    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:32.316562 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:32.316191    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:32.316562 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:32.316287    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:32.316562 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:32.316347    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:32.316562 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:32.316423    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:34.313812 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:34.313012    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:34.313812 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:34.313172    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:34.313812 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:34.313589    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:34.313812 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:34.313691    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:34.313812 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:34.313777    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:34.314532 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:34.313858    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:35.898579 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:35.898536    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:35.898579 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:35.898582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:35.899100 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.898697    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:35.899100 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.898794    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:43.898778082 +0000 UTC m=+18.088807816 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:35.899181 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.899146    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:35.899214 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.899182    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:43.899170843 +0000 UTC m=+18.089200577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:35.999672 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:35.999159    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:35.999672 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.999356    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:57:35.999672 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.999381    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:57:35.999672 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.999396    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:35.999672 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:35.999467    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:57:43.999446331 +0000 UTC m=+18.189476311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:36.314349 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:36.314042    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:36.314349 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:36.314186    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:36.314922 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:36.314623    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:36.314922 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:36.314726    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:36.314922 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:36.314804    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:36.314922 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:36.314880    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:38.313387 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:38.313352    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:38.313863 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:38.313499    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:38.313863 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:38.313831    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:38.313976 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:38.313922    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:38.313976 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:38.313952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:38.314513 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:38.314479    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:40.313594 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:40.313559    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:40.314085 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:40.313559    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:40.314085 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:40.313725    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:40.314085 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:40.313803    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:40.314085 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:40.313559    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:40.314085 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:40.313929    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:42.312927 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:42.312889    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:42.313357 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:42.312902    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:42.313357 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:42.313021    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:42.313357 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:42.313143    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:42.313357 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:42.312902    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:42.313357 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:42.313250    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:43.962180 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:43.962105    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:43.962758 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:43.962189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:43.962758 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:43.962281    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:43.962758 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:43.962350    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:43.962758 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:43.962357    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:59.962336081 +0000 UTC m=+34.152365829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:43.962758 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:43.962421    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:57:59.962402828 +0000 UTC m=+34.152432572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:44.063119 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:44.063087    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:44.063286 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.063262    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:57:44.063286 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.063283    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:57:44.063388 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.063293    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:44.063388 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.063342    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:58:00.063326779 +0000 UTC m=+34.253356735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:57:44.313406 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:44.313373    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:44.313581 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:44.313373    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:44.313581 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.313490    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:44.313581 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:44.313496    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:44.313581 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.313568    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:44.313925 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:44.313636    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:46.314483 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:46.314448    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:46.314945 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:46.314548    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:46.314945 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:46.314637    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:46.314945 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:46.314762    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:46.314945 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:46.314815    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:46.314945 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:46.314896    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:48.313638 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:48.313603    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:48.314132 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:48.313603    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:48.314132 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:48.313753    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:48.314132 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:48.313744    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:48.314132 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:48.313866    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:48.314132 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:48.313948    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:49.527071 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.526728    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rtr8b" event={"ID":"e3562f0e-f4c2-447d-a176-0596c7b205bd","Type":"ContainerStarted","Data":"b71101178be150d9db3f63bd7d6721c791b74715038b6695e905814253f4be39"}
Jul 26 17:57:49.528183 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.528161    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dtpb6" event={"ID":"9f0c8ff6-aa29-4b61-aed1-7e1535304b49","Type":"ContainerStarted","Data":"432e300a6bc02184ee98c01d668c7de1eb50e2b2bf27077cddc82b40f591b055"}
Jul 26 17:57:49.529597 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.529577    2565 generic.go:356] "Generic (PLEG): container finished" podID="06937257-8927-494e-a4a6-532585fdc8e8" containerID="a9747d707d5c8e7c915c4df63f1dd4344e6afbe17eca876cf3eb043ec8cbcd19" exitCode=0
Jul 26 17:57:49.529674 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.529653    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerDied","Data":"a9747d707d5c8e7c915c4df63f1dd4344e6afbe17eca876cf3eb043ec8cbcd19"}
Jul 26 17:57:49.531130 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.531103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pqvrw" event={"ID":"85696f9e-3458-467e-bf4b-9503978d99ca","Type":"ContainerStarted","Data":"9ae9a3bc964dc2be3da55ee271b5ee1fe65df90e24ec117ebb20d13086ad4bd0"}
Jul 26 17:57:49.532479 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.532457    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7jpg7" event={"ID":"17b413f2-3a11-4e6e-bd9e-2c72c576749d","Type":"ContainerStarted","Data":"b16330a52fd74f96c219c2175d3d56b0dde692fc1d6eb0f2d4e5d58ce0666693"}
Jul 26 17:57:49.533837 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.533804    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fkfch" event={"ID":"a50ffe58-a86d-46f0-9feb-5d96623d19db","Type":"ContainerStarted","Data":"d7cf69f168a7460382eab0b3c70bd795a1256581da1125f082c56152c09766f2"}
Jul 26 17:57:49.535114 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.535096    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954" event={"ID":"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096","Type":"ContainerStarted","Data":"d56042e85152a8c9061ecd37449efb0ea8a55da828ffc7d8bac4d5193a7cb97f"}
Jul 26 17:57:49.538322 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.538303    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"c2a8800c0d0227053557fa250697c49a55d9f370d3d68be0202b8fb041707db3"}
Jul 26 17:57:49.538429 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.538373    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"e71efc8c2a4b95be19dadbfd2ffd45fb81ee064601b5d5ea0a513a2dbf630e48"}
Jul 26 17:57:49.538429 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.538425    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"937e05ea4c8aacc6817d23cbbfc7c54ef7d80867623c529c3682db02d5dbb626"}
Jul 26 17:57:49.538543 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.538472    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"a1b34b4bc8365caa57559cad566589655d28207c8d2dabead6dcd4fe50da7114"}
Jul 26 17:57:49.538543 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.538520    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"00b8d117de0eb19e26afaaa9ae0dbe5be7d2393b58a2aa85699458692b4823b5"}
Jul 26 17:57:49.538633 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.538569    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"cbf9b3c0f16e7a87ce348f53d7abfe89a691247d124f270707d9c7859df3965a"}
Jul 26 17:57:49.543461 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.543424    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rtr8b" podStartSLOduration=3.457129022 podStartE2EDuration="22.543413585s" totalImagesPullingTime="19.086284563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:49.543256181 +0000 UTC m=+23.733285935" watchObservedRunningTime="2026-07-26 17:57:49.543413585 +0000 UTC m=+23.733443338"
Jul 26 17:57:49.543758 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.543731    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-34.ec2.internal" podStartSLOduration=21.543725133 podStartE2EDuration="21.543725133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:31.489897665 +0000 UTC m=+5.679927424" watchObservedRunningTime="2026-07-26 17:57:49.543725133 +0000 UTC m=+23.733754886"
Jul 26 17:57:49.560300 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.560261    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dtpb6" podStartSLOduration=3.528062265 podStartE2EDuration="22.560248846s" totalImagesPullingTime="19.032186581s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:49.560200106 +0000 UTC m=+23.750229861" watchObservedRunningTime="2026-07-26 17:57:49.560248846 +0000 UTC m=+23.750278600"
Jul 26 17:57:49.596787 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.596750    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fkfch" podStartSLOduration=11.399365748 podStartE2EDuration="22.596736276s" totalImagesPullingTime="11.197370528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:49.596519068 +0000 UTC m=+23.786548822" watchObservedRunningTime="2026-07-26 17:57:49.596736276 +0000 UTC m=+23.786766052"
Jul 26 17:57:49.634432 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.634388    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7jpg7" podStartSLOduration=3.60794958 podStartE2EDuration="22.634372145s" totalImagesPullingTime="19.026422565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:49.611941647 +0000 UTC m=+23.801971400" watchObservedRunningTime="2026-07-26 17:57:49.634372145 +0000 UTC m=+23.824401899"
Jul 26 17:57:49.634730 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.634690    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pqvrw" podStartSLOduration=3.6071936879999997 podStartE2EDuration="22.63468463s" totalImagesPullingTime="19.027490942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:49.633760685 +0000 UTC m=+23.823790439" watchObservedRunningTime="2026-07-26 17:57:49.63468463 +0000 UTC m=+23.824714383"
Jul 26 17:57:49.862376 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:49.862354    2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 17:57:50.196363 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.196212    2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T17:57:49.862371763Z","UUID":"02a5a67f-f760-4d1c-9f81-046f84fe5ea7","Handler":null,"Name":"","Endpoint":""}
Jul 26 17:57:50.198033 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.198008    2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 17:57:50.198158 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.198055    2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 17:57:50.313444 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.313417    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:50.313618 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.313503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:50.313618 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.313417    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:50.313755 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:50.313628    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:50.313817 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:50.313747    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:50.313817 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:50.313785    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:50.542797 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.542758    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954" event={"ID":"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096","Type":"ContainerStarted","Data":"db3b780741fde0656377687e317f9ede8b99a5b4206f2a176ab074879d32d32d"}
Jul 26 17:57:50.544722 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.544341    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tfpnp" event={"ID":"cb7571f6-7ed7-4b3e-b331-b0855c53f528","Type":"ContainerStarted","Data":"f004a003ea1dc6a1aae0020aafec1e9a503bb9912c13f96f4c20864c4f8422ea"}
Jul 26 17:57:50.560496 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:50.560004    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tfpnp" podStartSLOduration=4.52068923 podStartE2EDuration="23.559992461s" totalImagesPullingTime="19.039303231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:50.559626933 +0000 UTC m=+24.749656687" watchObservedRunningTime="2026-07-26 17:57:50.559992461 +0000 UTC m=+24.750022214"
Jul 26 17:57:51.181630 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.181608    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:51.182216 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.182193    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:51.549657 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.549625    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954" event={"ID":"6a3ca5ad-c732-4b8c-bc71-ddbf59eff096","Type":"ContainerStarted","Data":"879c2138589ad93764c2f49b7ce529990030db7ce4f2817febfdf5f59b1122b5"}
Jul 26 17:57:51.553488 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.553445    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"c6da9d8efb21b9a79135d37a0da56632418d422bc34373c9a345548e8b71ba08"}
Jul 26 17:57:51.553874 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.553812    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:51.554252 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.554233    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fkfch"
Jul 26 17:57:51.567504 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:51.567462    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gl954" podStartSLOduration=3.494290423 podStartE2EDuration="24.567446578s" totalImagesPullingTime="21.073156155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:51.567173953 +0000 UTC m=+25.757203711" watchObservedRunningTime="2026-07-26 17:57:51.567446578 +0000 UTC m=+25.757476332"
Jul 26 17:57:52.313243 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:52.313010    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:52.313466 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:52.313018    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:52.313466 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:52.313345    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:52.313466 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:52.313084    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:52.313466 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:52.313426    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:52.313714 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:52.313523    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:53.561154 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:53.561129    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" event={"ID":"66f1b080-76d2-4286-912b-cb3d61403c70","Type":"ContainerStarted","Data":"f66e8d3e783eb469f84b54663b75525cd864f6b843bbdf3cf26699c53b85d3b6"}
Jul 26 17:57:53.561731 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:53.561463    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:53.561731 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:53.561527    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:53.576731 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:53.576571    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:53.605751 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:53.605690    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr" podStartSLOduration=7.507277286 podStartE2EDuration="26.605676281s" totalImagesPullingTime="19.098398995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:57:53.60313561 +0000 UTC m=+27.793165374" watchObservedRunningTime="2026-07-26 17:57:53.605676281 +0000 UTC m=+27.795706035"
Jul 26 17:57:54.316592 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.316564    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:54.316769 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.316568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:54.316769 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.316568    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:54.316769 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:54.316669    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:54.316890 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:54.316780    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:54.316890 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:54.316848    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:54.565197 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.565168    2565 generic.go:356] "Generic (PLEG): container finished" podID="06937257-8927-494e-a4a6-532585fdc8e8" containerID="d6ccdbed2bfef5e23a8c49a1162fdac9387492e1d293f5fd23e1e3bb7e279550" exitCode=0
Jul 26 17:57:54.565590 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.565257    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerDied","Data":"d6ccdbed2bfef5e23a8c49a1162fdac9387492e1d293f5fd23e1e3bb7e279550"}
Jul 26 17:57:54.565741 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.565718    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:54.581713 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:54.581671    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:57:55.458534 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.458492    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4k55t"]
Jul 26 17:57:55.458678 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.458633    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:55.458755 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:55.458737    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:55.463354 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.463328    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n68jr"]
Jul 26 17:57:55.463478 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.463427    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:55.463739 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:55.463521    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:55.464031 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.464009    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nthlf"]
Jul 26 17:57:55.464135 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.464120    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:55.464218 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:55.464200    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:55.569680 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.569638    2565 generic.go:356] "Generic (PLEG): container finished" podID="06937257-8927-494e-a4a6-532585fdc8e8" containerID="083f7016f3f927fddba136385cfc83c36fc9cceeeb82e831ad1d145c7bea62c8" exitCode=0
Jul 26 17:57:55.570075 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:55.569727    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerDied","Data":"083f7016f3f927fddba136385cfc83c36fc9cceeeb82e831ad1d145c7bea62c8"}
Jul 26 17:57:57.313865 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:57.313825    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:57.314389 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:57.313944    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:57.314389 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:57.313825    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:57.314389 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:57.313825    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:57.314389 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:57.314058    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:57.314389 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:57.314170    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:57.577070 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:57.576976    2565 generic.go:356] "Generic (PLEG): container finished" podID="06937257-8927-494e-a4a6-532585fdc8e8" containerID="41c62e737ec622124e1f60f5fd6d7bf3437b5eb3f6768a090a4417b2856d4dff" exitCode=0
Jul 26 17:57:57.577070 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:57.577015    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerDied","Data":"41c62e737ec622124e1f60f5fd6d7bf3437b5eb3f6768a090a4417b2856d4dff"}
Jul 26 17:57:59.312991 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:59.312952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:59.312991 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:59.312992    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:57:59.313831 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:59.313072    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:59.313831 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.313091    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nthlf" podUID="caba9223-8c40-4e7e-9740-98fcb97755b0"
Jul 26 17:57:59.313831 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.313163    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4k55t" podUID="3b6e1a55-1afc-40a7-be4c-9f5810802717"
Jul 26 17:57:59.313831 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.313214    2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n68jr" podUID="26478850-8053-48df-b800-bff475488a1e"
Jul 26 17:57:59.990253 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:59.990157    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:57:59.990253 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:57:59.990202    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:57:59.990477 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.990341    2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:59.990477 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.990344    2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:57:59.990477 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.990411    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret podName:3b6e1a55-1afc-40a7-be4c-9f5810802717 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:31.99039183 +0000 UTC m=+66.180421566 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret") pod "global-pull-secret-syncer-4k55t" (UID: "3b6e1a55-1afc-40a7-be4c-9f5810802717") : object "kube-system"/"original-pull-secret" not registered
Jul 26 17:57:59.990477 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:57:59.990429    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs podName:caba9223-8c40-4e7e-9740-98fcb97755b0 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:31.990420282 +0000 UTC m=+66.180450015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs") pod "network-metrics-daemon-nthlf" (UID: "caba9223-8c40-4e7e-9740-98fcb97755b0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 17:58:00.091319 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:00.091284    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:58:00.091469 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:00.091451    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 17:58:00.091510 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:00.091473    2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 17:58:00.091510 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:00.091484    2565 projected.go:196] Error preparing data for projected volume kube-api-access-tm8ps for pod openshift-network-diagnostics/network-check-target-n68jr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:58:00.091571 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:00.091534    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps podName:26478850-8053-48df-b800-bff475488a1e nodeName:}" failed. No retries permitted until 2026-07-26 17:58:32.091521273 +0000 UTC m=+66.281551004 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tm8ps" (UniqueName: "kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps") pod "network-check-target-n68jr" (UID: "26478850-8053-48df-b800-bff475488a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 17:58:01.131759 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.131730    2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-34.ec2.internal" event="NodeReady"
Jul 26 17:58:01.132289 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.131891    2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 17:58:01.177579 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.177548    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rlbpg"]
Jul 26 17:58:01.202258 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.202183    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6jtsk"]
Jul 26 17:58:01.202392 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.202337    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.205766 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.205267    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 17:58:01.205766 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.205405    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-566xh\""
Jul 26 17:58:01.205766 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.205468    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 17:58:01.216258 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.216232    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rlbpg"]
Jul 26 17:58:01.216357 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.216277    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6jtsk"]
Jul 26 17:58:01.216405 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.216372    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.219831 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.219808    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 17:58:01.219934 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.219812    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 17:58:01.220079 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.220058    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k6gz2\""
Jul 26 17:58:01.220194 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.220099    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 17:58:01.299027 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.298996    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r724f\" (UniqueName: \"kubernetes.io/projected/23eee869-252e-495d-9a41-efaf13e2e3e6-kube-api-access-r724f\") pod \"ingress-canary-6jtsk\" (UID: \"23eee869-252e-495d-9a41-efaf13e2e3e6\") " pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.299224 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.299043    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brj8z\" (UniqueName: \"kubernetes.io/projected/9e52b638-70d2-4be3-99f0-0f5b36322b7f-kube-api-access-brj8z\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.299224 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.299112    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e52b638-70d2-4be3-99f0-0f5b36322b7f-tmp-dir\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.299224 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.299149    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.299224 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.299191    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23eee869-252e-495d-9a41-efaf13e2e3e6-cert\") pod \"ingress-canary-6jtsk\" (UID: \"23eee869-252e-495d-9a41-efaf13e2e3e6\") " pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.299392 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.299231    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e52b638-70d2-4be3-99f0-0f5b36322b7f-config-volume\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.313150 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.313114    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:58:01.313150 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.313138    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:58:01.313338 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.313155    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:58:01.316378 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.316354    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 17:58:01.316531 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.316403    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 17:58:01.316531 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.316427    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9fbzz\""
Jul 26 17:58:01.316531 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.316478    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 17:58:01.316682 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.316622    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7vmj\""
Jul 26 17:58:01.316682 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.316644    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 17:58:01.400154 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400112    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.400327 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400177    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23eee869-252e-495d-9a41-efaf13e2e3e6-cert\") pod \"ingress-canary-6jtsk\" (UID: \"23eee869-252e-495d-9a41-efaf13e2e3e6\") " pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.400327 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400213    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e52b638-70d2-4be3-99f0-0f5b36322b7f-config-volume\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.400327 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:01.400228    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:58:01.400327 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400265    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r724f\" (UniqueName: \"kubernetes.io/projected/23eee869-252e-495d-9a41-efaf13e2e3e6-kube-api-access-r724f\") pod \"ingress-canary-6jtsk\" (UID: \"23eee869-252e-495d-9a41-efaf13e2e3e6\") " pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.400327 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:01.400307    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls podName:9e52b638-70d2-4be3-99f0-0f5b36322b7f nodeName:}" failed. No retries permitted until 2026-07-26 17:58:01.900287584 +0000 UTC m=+36.090317328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls") pod "dns-default-rlbpg" (UID: "9e52b638-70d2-4be3-99f0-0f5b36322b7f") : secret "dns-default-metrics-tls" not found
Jul 26 17:58:01.400327 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400327    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brj8z\" (UniqueName: \"kubernetes.io/projected/9e52b638-70d2-4be3-99f0-0f5b36322b7f-kube-api-access-brj8z\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.400643 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400367    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e52b638-70d2-4be3-99f0-0f5b36322b7f-tmp-dir\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.400690 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400677    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e52b638-70d2-4be3-99f0-0f5b36322b7f-tmp-dir\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.400839 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.400812    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9e52b638-70d2-4be3-99f0-0f5b36322b7f-config-volume\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.405328 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.405306    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23eee869-252e-495d-9a41-efaf13e2e3e6-cert\") pod \"ingress-canary-6jtsk\" (UID: \"23eee869-252e-495d-9a41-efaf13e2e3e6\") " pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.409261 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.409232    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brj8z\" (UniqueName: \"kubernetes.io/projected/9e52b638-70d2-4be3-99f0-0f5b36322b7f-kube-api-access-brj8z\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.410497 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.410464    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r724f\" (UniqueName: \"kubernetes.io/projected/23eee869-252e-495d-9a41-efaf13e2e3e6-kube-api-access-r724f\") pod \"ingress-canary-6jtsk\" (UID: \"23eee869-252e-495d-9a41-efaf13e2e3e6\") " pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.525445 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.525408    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6jtsk"
Jul 26 17:58:01.810515 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.810276    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6jtsk"]
Jul 26 17:58:01.903124 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:01.903083    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:01.903294 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:01.903226    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:58:01.903345 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:01.903297    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls podName:9e52b638-70d2-4be3-99f0-0f5b36322b7f nodeName:}" failed. No retries permitted until 2026-07-26 17:58:02.903278706 +0000 UTC m=+37.093308437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls") pod "dns-default-rlbpg" (UID: "9e52b638-70d2-4be3-99f0-0f5b36322b7f") : secret "dns-default-metrics-tls" not found
Jul 26 17:58:02.590482 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:02.590444    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6jtsk" event={"ID":"23eee869-252e-495d-9a41-efaf13e2e3e6","Type":"ContainerStarted","Data":"1175437f8a0bea585121eda8db3e94db442d07ae22952c83b3a133b0065eac24"}
Jul 26 17:58:02.593874 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:02.593845    2565 generic.go:356] "Generic (PLEG): container finished" podID="06937257-8927-494e-a4a6-532585fdc8e8" containerID="cbcfb3908f78528d6328e1765f5786b3ff5e6db6d858a6b97d391067cefc724a" exitCode=0
Jul 26 17:58:02.593991 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:02.593887    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerDied","Data":"cbcfb3908f78528d6328e1765f5786b3ff5e6db6d858a6b97d391067cefc724a"}
Jul 26 17:58:02.910977 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:02.910897    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:02.911141 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:02.911063    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:58:02.911206 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:02.911152    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls podName:9e52b638-70d2-4be3-99f0-0f5b36322b7f nodeName:}" failed. No retries permitted until 2026-07-26 17:58:04.911129345 +0000 UTC m=+39.101159077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls") pod "dns-default-rlbpg" (UID: "9e52b638-70d2-4be3-99f0-0f5b36322b7f") : secret "dns-default-metrics-tls" not found
Jul 26 17:58:03.361162 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.361130    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"]
Jul 26 17:58:03.364739 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.364716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"
Jul 26 17:58:03.367590 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.367567    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 26 17:58:03.367715 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.367630    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 26 17:58:03.367792 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.367771    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xhlxt\""
Jul 26 17:58:03.374554 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.374533    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"]
Jul 26 17:58:03.416135 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.416061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmhlw\" (UniqueName: \"kubernetes.io/projected/568409b6-709f-4564-b94a-4aa912e208e6-kube-api-access-gmhlw\") pod \"migrator-6865c9967-tp9dq\" (UID: \"568409b6-709f-4564-b94a-4aa912e208e6\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"
Jul 26 17:58:03.516978 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.516940    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmhlw\" (UniqueName: \"kubernetes.io/projected/568409b6-709f-4564-b94a-4aa912e208e6-kube-api-access-gmhlw\") pod \"migrator-6865c9967-tp9dq\" (UID: \"568409b6-709f-4564-b94a-4aa912e208e6\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"
Jul 26 17:58:03.527755 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.527721    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmhlw\" (UniqueName: \"kubernetes.io/projected/568409b6-709f-4564-b94a-4aa912e208e6-kube-api-access-gmhlw\") pod \"migrator-6865c9967-tp9dq\" (UID: \"568409b6-709f-4564-b94a-4aa912e208e6\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"
Jul 26 17:58:03.599819 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.599785    2565 generic.go:356] "Generic (PLEG): container finished" podID="06937257-8927-494e-a4a6-532585fdc8e8" containerID="b40d1e846ace4a38feac40a2d8b9ecdb176e9b0a7776f4034aed71e09a9b6e66" exitCode=0
Jul 26 17:58:03.600255 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.599857    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerDied","Data":"b40d1e846ace4a38feac40a2d8b9ecdb176e9b0a7776f4034aed71e09a9b6e66"}
Jul 26 17:58:03.675026 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:03.674946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"
Jul 26 17:58:04.107563 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.107403    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq"]
Jul 26 17:58:04.110994 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:04.110961    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568409b6_709f_4564_b94a_4aa912e208e6.slice/crio-1252871d14508c262f0513172e08e3346a5d3c7b9394a07218520bce11859b1a WatchSource:0}: Error finding container 1252871d14508c262f0513172e08e3346a5d3c7b9394a07218520bce11859b1a: Status 404 returned error can't find the container with id 1252871d14508c262f0513172e08e3346a5d3c7b9394a07218520bce11859b1a
Jul 26 17:58:04.291140 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.291067    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kr997"]
Jul 26 17:58:04.296147 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.296117    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.298803 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.298784    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2d24l\""
Jul 26 17:58:04.298902 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.298783    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 17:58:04.299220 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.299198    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 17:58:04.299331 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.299314    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 17:58:04.299395 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.299346    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 17:58:04.304065 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.304046    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kr997"]
Jul 26 17:58:04.322930 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.322905    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6eff04b4-327d-4f73-b801-698564387ab4-data-volume\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.323064 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.322942    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6eff04b4-327d-4f73-b801-698564387ab4-crio-socket\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.323064 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.322962    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.323064 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.322993    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbdlj\" (UniqueName: \"kubernetes.io/projected/6eff04b4-327d-4f73-b801-698564387ab4-kube-api-access-fbdlj\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.323064 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.323032    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423490 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423459    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423667 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423508    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6eff04b4-327d-4f73-b801-698564387ab4-data-volume\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423667 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423534    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6eff04b4-327d-4f73-b801-698564387ab4-crio-socket\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423667 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423559    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423667 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423580    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbdlj\" (UniqueName: \"kubernetes.io/projected/6eff04b4-327d-4f73-b801-698564387ab4-kube-api-access-fbdlj\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423667 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.423590    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:58:04.423667 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.423659    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:04.423920 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.423674    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:04.923652156 +0000 UTC m=+39.113681892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : configmap "kube-rbac-proxy" not found
Jul 26 17:58:04.423920 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.423725    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:04.923691404 +0000 UTC m=+39.113721149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:04.423920 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423801    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6eff04b4-327d-4f73-b801-698564387ab4-crio-socket\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.423920 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.423850    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6eff04b4-327d-4f73-b801-698564387ab4-data-volume\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.433628 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.433610    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbdlj\" (UniqueName: \"kubernetes.io/projected/6eff04b4-327d-4f73-b801-698564387ab4-kube-api-access-fbdlj\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.605360 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.605269    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" event={"ID":"06937257-8927-494e-a4a6-532585fdc8e8","Type":"ContainerStarted","Data":"065d90211f7cd9702cd95d6e3b4c25c91c04d9347aec5e9d5d7f4907391a2f3a"}
Jul 26 17:58:04.606724 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.606683    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6jtsk" event={"ID":"23eee869-252e-495d-9a41-efaf13e2e3e6","Type":"ContainerStarted","Data":"957d85cd5917ba5e18d5aa4fc51bc6d6460cb22980736cd040f6b50d2a0bb1da"}
Jul 26 17:58:04.607668 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.607646    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq" event={"ID":"568409b6-709f-4564-b94a-4aa912e208e6","Type":"ContainerStarted","Data":"1252871d14508c262f0513172e08e3346a5d3c7b9394a07218520bce11859b1a"}
Jul 26 17:58:04.630148 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.630111    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ljtcc" podStartSLOduration=8.252207604 podStartE2EDuration="37.630096776s" totalImagesPullingTime="29.377889172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:04.629616961 +0000 UTC m=+38.819646715" watchObservedRunningTime="2026-07-26 17:58:04.630096776 +0000 UTC m=+38.820126581"
Jul 26 17:58:04.648472 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.648427    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6jtsk" podStartSLOduration=1.534584528 podStartE2EDuration="3.648412963s" totalImagesPullingTime="2.113828435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:04.647244702 +0000 UTC m=+38.837274447" watchObservedRunningTime="2026-07-26 17:58:04.648412963 +0000 UTC m=+38.838442716"
Jul 26 17:58:04.928059 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.927976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.928059 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.928046    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:04.928233 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:04.928074    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:04.928233 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.928136    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:04.928233 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.928169    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:58:04.928233 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.928201    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:05.928182974 +0000 UTC m=+40.118212728 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:04.928233 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.928227    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:05.928212423 +0000 UTC m=+40.118242160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : configmap "kube-rbac-proxy" not found
Jul 26 17:58:04.928233 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.928177    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:58:04.928416 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:04.928254    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls podName:9e52b638-70d2-4be3-99f0-0f5b36322b7f nodeName:}" failed. No retries permitted until 2026-07-26 17:58:08.928248991 +0000 UTC m=+43.118278723 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls") pod "dns-default-rlbpg" (UID: "9e52b638-70d2-4be3-99f0-0f5b36322b7f") : secret "dns-default-metrics-tls" not found
Jul 26 17:58:05.611910 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:05.611655    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq" event={"ID":"568409b6-709f-4564-b94a-4aa912e208e6","Type":"ContainerStarted","Data":"4202d2601b9abcefd9443fa6a00d28fbe34b056b6fef4f72f9935807cdeab089"}
Jul 26 17:58:05.611910 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:05.611881    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq" event={"ID":"568409b6-709f-4564-b94a-4aa912e208e6","Type":"ContainerStarted","Data":"165eec25ea6c011ba2657f4775d2e040e65820e4deddacfe1a7e639cd814f1f0"}
Jul 26 17:58:05.630540 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:05.630490    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-tp9dq" podStartSLOduration=1.4852519069999999 podStartE2EDuration="2.630474309s" totalImagesPullingTime="1.145222402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:05.629819461 +0000 UTC m=+39.819849215" watchObservedRunningTime="2026-07-26 17:58:05.630474309 +0000 UTC m=+39.820504063"
Jul 26 17:58:05.936015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:05.935904    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:05.936015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:05.935991    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:05.936263 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:05.936074    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:05.936263 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:05.936145    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:07.93612704 +0000 UTC m=+42.126156779 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:05.936263 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:05.936082    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:58:05.936263 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:05.936210    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:07.936196842 +0000 UTC m=+42.126226573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : configmap "kube-rbac-proxy" not found
Jul 26 17:58:06.896797 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:06.896771    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7jpg7_17b413f2-3a11-4e6e-bd9e-2c72c576749d/dns-node-resolver/0.log"
Jul 26 17:58:07.948947 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:07.948907    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:07.949393 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:07.948997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:07.949393 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:07.949024    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:58:07.949393 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:07.949087    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:11.949072083 +0000 UTC m=+46.139101814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : configmap "kube-rbac-proxy" not found
Jul 26 17:58:07.949393 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:07.949114    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:07.949393 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:07.949165    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:11.949149055 +0000 UTC m=+46.139178787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:08.298454 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:08.298427    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pqvrw_85696f9e-3458-467e-bf4b-9503978d99ca/node-ca/0.log"
Jul 26 17:58:08.698023 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:08.697956    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6jtsk_23eee869-252e-495d-9a41-efaf13e2e3e6/serve-healthcheck-canary/0.log"
Jul 26 17:58:08.956582 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:08.956505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:08.956907 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:08.956629    2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 26 17:58:08.956907 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:08.956679    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls podName:9e52b638-70d2-4be3-99f0-0f5b36322b7f nodeName:}" failed. No retries permitted until 2026-07-26 17:58:16.956665286 +0000 UTC m=+51.146695018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls") pod "dns-default-rlbpg" (UID: "9e52b638-70d2-4be3-99f0-0f5b36322b7f") : secret "dns-default-metrics-tls" not found
Jul 26 17:58:09.297085 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:09.297055    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tp9dq_568409b6-709f-4564-b94a-4aa912e208e6/migrator/0.log"
Jul 26 17:58:09.498063 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:09.498037    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tp9dq_568409b6-709f-4564-b94a-4aa912e208e6/graceful-termination/0.log"
Jul 26 17:58:11.978598 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:11.978561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:11.979160 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:11.978633    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:11.979160 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:11.978729    2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 26 17:58:11.979160 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:11.978754    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:11.979160 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:11.978808    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:19.978788974 +0000 UTC m=+54.168818708 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:11.979160 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:11.978824    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:19.978815974 +0000 UTC m=+54.168845705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : configmap "kube-rbac-proxy" not found
Jul 26 17:58:17.016921 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:17.016878    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:17.019810 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:17.019787    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e52b638-70d2-4be3-99f0-0f5b36322b7f-metrics-tls\") pod \"dns-default-rlbpg\" (UID: \"9e52b638-70d2-4be3-99f0-0f5b36322b7f\") " pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:17.113388 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:17.113351    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:17.277020 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:17.276841    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rlbpg"]
Jul 26 17:58:17.279353 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:17.279330    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e52b638_70d2_4be3_99f0_0f5b36322b7f.slice/crio-026439b1349a128947600fea7d06d282e620450129c8ccd5d421bac770c7f208 WatchSource:0}: Error finding container 026439b1349a128947600fea7d06d282e620450129c8ccd5d421bac770c7f208: Status 404 returned error can't find the container with id 026439b1349a128947600fea7d06d282e620450129c8ccd5d421bac770c7f208
Jul 26 17:58:17.644141 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:17.641583    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rlbpg" event={"ID":"9e52b638-70d2-4be3-99f0-0f5b36322b7f","Type":"ContainerStarted","Data":"026439b1349a128947600fea7d06d282e620450129c8ccd5d421bac770c7f208"}
Jul 26 17:58:20.038532 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.038501    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:20.038875 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.038560    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:20.038875 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:20.038651    2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:20.038875 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:20.038730    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls podName:6eff04b4-327d-4f73-b801-698564387ab4 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:36.03871197 +0000 UTC m=+70.228741717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kr997" (UID: "6eff04b4-327d-4f73-b801-698564387ab4") : secret "insights-runtime-extractor-tls" not found
Jul 26 17:58:20.039082 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.039064    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6eff04b4-327d-4f73-b801-698564387ab4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:20.650591 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.650560    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rlbpg" event={"ID":"9e52b638-70d2-4be3-99f0-0f5b36322b7f","Type":"ContainerStarted","Data":"e51d04530cfb39284de221e3baa259b0bc746618e180336afdd1e32f926e3535"}
Jul 26 17:58:20.650776 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.650623    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rlbpg" event={"ID":"9e52b638-70d2-4be3-99f0-0f5b36322b7f","Type":"ContainerStarted","Data":"aaec4b5db6c414c8d44c735cc37d55d61d56d4b296b5951123b25b7f4d03aa69"}
Jul 26 17:58:20.650776 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.650658    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:20.667248 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:20.667195    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rlbpg" podStartSLOduration=17.216836782 podStartE2EDuration="19.667179998s" totalImagesPullingTime="2.450343216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:20.666375269 +0000 UTC m=+54.856405024" watchObservedRunningTime="2026-07-26 17:58:20.667179998 +0000 UTC m=+54.857209751"
Jul 26 17:58:25.733124 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.733091    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"]
Jul 26 17:58:25.736405 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.736388    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.740441 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.740419    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"]
Jul 26 17:58:25.742820 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.742802    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.747994 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:25.747969    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-service-proxy-server-certificates\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" type="*v1.Secret"
Jul 26 17:58:25.747994 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.747985    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 26 17:58:25.748113 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:25.747985    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" type="*v1.Secret"
Jul 26 17:58:25.748848 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:25.748828    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" type="*v1.Secret"
Jul 26 17:58:25.748999 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:25.748982    2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"cluster-proxy-ca\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" type="*v1.Secret"
Jul 26 17:58:25.748999 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.748988    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 26 17:58:25.749191 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.749179    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7wp2z\""
Jul 26 17:58:25.749965 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.749952    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 26 17:58:25.751905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.751888    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"]
Jul 26 17:58:25.767655 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.767637    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 26 17:58:25.777836 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.777814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-ca\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.777905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.777839    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.777905 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.777869    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7f7142d-3e6e-4712-9951-eb9fb0787faa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65876766d4-tdrj6\" (UID: \"b7f7142d-3e6e-4712-9951-eb9fb0787faa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.777989 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.777913    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlqxn\" (UniqueName: \"kubernetes.io/projected/b7f7142d-3e6e-4712-9951-eb9fb0787faa-kube-api-access-xlqxn\") pod \"managed-serviceaccount-addon-agent-65876766d4-tdrj6\" (UID: \"b7f7142d-3e6e-4712-9951-eb9fb0787faa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.778044 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.778005    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.778097 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.778051    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.778097 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.778084    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c67e896-4b1d-4684-9566-6811a410c830-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.778199 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.778110    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdmd\" (UniqueName: \"kubernetes.io/projected/5c67e896-4b1d-4684-9566-6811a410c830-kube-api-access-2cdmd\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.804857 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.804831    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"]
Jul 26 17:58:25.878852 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.878824    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7f7142d-3e6e-4712-9951-eb9fb0787faa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65876766d4-tdrj6\" (UID: \"b7f7142d-3e6e-4712-9951-eb9fb0787faa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.879015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.878869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlqxn\" (UniqueName: \"kubernetes.io/projected/b7f7142d-3e6e-4712-9951-eb9fb0787faa-kube-api-access-xlqxn\") pod \"managed-serviceaccount-addon-agent-65876766d4-tdrj6\" (UID: \"b7f7142d-3e6e-4712-9951-eb9fb0787faa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.879015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.878890    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.879015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.878923    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.879015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.878949    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c67e896-4b1d-4684-9566-6811a410c830-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.879015 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.878976    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdmd\" (UniqueName: \"kubernetes.io/projected/5c67e896-4b1d-4684-9566-6811a410c830-kube-api-access-2cdmd\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.879277 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.879107    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-ca\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.879277 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.879161    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.879835 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.879814    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5c67e896-4b1d-4684-9566-6811a410c830-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:25.881289 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.881272    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b7f7142d-3e6e-4712-9951-eb9fb0787faa-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65876766d4-tdrj6\" (UID: \"b7f7142d-3e6e-4712-9951-eb9fb0787faa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.907029 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.907001    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlqxn\" (UniqueName: \"kubernetes.io/projected/b7f7142d-3e6e-4712-9951-eb9fb0787faa-kube-api-access-xlqxn\") pod \"managed-serviceaccount-addon-agent-65876766d4-tdrj6\" (UID: \"b7f7142d-3e6e-4712-9951-eb9fb0787faa\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:25.917078 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:25.917055    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdmd\" (UniqueName: \"kubernetes.io/projected/5c67e896-4b1d-4684-9566-6811a410c830-kube-api-access-2cdmd\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:26.055468 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.055370    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"
Jul 26 17:58:26.215831 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.215810    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6"]
Jul 26 17:58:26.217750 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:26.217687    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb7f7142d_3e6e_4712_9951_eb9fb0787faa.slice/crio-2a6bbe61b04be44f68d8fdbac70abbb89dd50cd4983ea879d6278767492f6873 WatchSource:0}: Error finding container 2a6bbe61b04be44f68d8fdbac70abbb89dd50cd4983ea879d6278767492f6873: Status 404 returned error can't find the container with id 2a6bbe61b04be44f68d8fdbac70abbb89dd50cd4983ea879d6278767492f6873
Jul 26 17:58:26.582525 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.582499    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cvvsr"
Jul 26 17:58:26.667061 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.667027    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6" event={"ID":"b7f7142d-3e6e-4712-9951-eb9fb0787faa","Type":"ContainerStarted","Data":"2a6bbe61b04be44f68d8fdbac70abbb89dd50cd4983ea879d6278767492f6873"}
Jul 26 17:58:26.794238 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.794208    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 26 17:58:26.801900 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.801878    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-ca\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:26.875751 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.875663    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 26 17:58:26.879353 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:26.879327    2565 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-hub-kubeconfig: failed to sync secret cache: timed out waiting for the condition
Jul 26 17:58:26.879488 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:26.879333    2565 secret.go:189] Couldn't get secret open-cluster-management-agent-addon/cluster-proxy-service-proxy-server-certificates: failed to sync secret cache: timed out waiting for the condition
Jul 26 17:58:26.879488 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:26.879402    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub-kubeconfig podName:5c67e896-4b1d-4684-9566-6811a410c830 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:27.379383659 +0000 UTC m=+61.569413393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "hub-kubeconfig" (UniqueName: "kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub-kubeconfig") pod "cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" (UID: "5c67e896-4b1d-4684-9566-6811a410c830") : failed to sync secret cache: timed out waiting for the condition
Jul 26 17:58:26.879488 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:26.879434    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-service-proxy-server-cert podName:5c67e896-4b1d-4684-9566-6811a410c830 nodeName:}" failed. No retries permitted until 2026-07-26 17:58:27.379416442 +0000 UTC m=+61.569446194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-proxy-server-cert" (UniqueName: "kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-service-proxy-server-cert") pod "cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" (UID: "5c67e896-4b1d-4684-9566-6811a410c830") : failed to sync secret cache: timed out waiting for the condition
Jul 26 17:58:26.881452 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.881434    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:26.956833 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:26.956800    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 26 17:58:27.124540 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.124504    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 26 17:58:27.389753 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.389713    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:27.389937 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.389860    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:27.392799 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.392746    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:27.392967 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.392851    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c67e896-4b1d-4684-9566-6811a410c830-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dbddcfdc6-jw66f\" (UID: \"5c67e896-4b1d-4684-9566-6811a410c830\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:27.561820 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.561755    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"
Jul 26 17:58:27.704716 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:27.704678    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f"]
Jul 26 17:58:28.725845 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:28.725816    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c67e896_4b1d_4684_9566_6811a410c830.slice/crio-b783bd86048f53a0f9d06a2c5936254aed8c3ffe02972ea082401e8d19524556 WatchSource:0}: Error finding container b783bd86048f53a0f9d06a2c5936254aed8c3ffe02972ea082401e8d19524556: Status 404 returned error can't find the container with id b783bd86048f53a0f9d06a2c5936254aed8c3ffe02972ea082401e8d19524556
Jul 26 17:58:29.678760 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:29.678719    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6" event={"ID":"b7f7142d-3e6e-4712-9951-eb9fb0787faa","Type":"ContainerStarted","Data":"212b55ca157c15c87ac99cf6adb1923588afe17c9bf7983b141f5fa779164bf2"}
Jul 26 17:58:29.680030 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:29.679993    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" event={"ID":"5c67e896-4b1d-4684-9566-6811a410c830","Type":"ContainerStarted","Data":"b783bd86048f53a0f9d06a2c5936254aed8c3ffe02972ea082401e8d19524556"}
Jul 26 17:58:29.697339 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:29.697038    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65876766d4-tdrj6" podStartSLOduration=2.135532523 podStartE2EDuration="4.697019145s" totalImagesPullingTime="2.561486622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:29.69666569 +0000 UTC m=+63.886695456" watchObservedRunningTime="2026-07-26 17:58:29.697019145 +0000 UTC m=+63.887048901"
Jul 26 17:58:30.656210 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:30.656182    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rlbpg"
Jul 26 17:58:32.029770 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.029729    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:58:32.030247 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.029869    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:58:32.032557 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.032492    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 17:58:32.033646 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.033620    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 17:58:32.042832 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.042803    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caba9223-8c40-4e7e-9740-98fcb97755b0-metrics-certs\") pod \"network-metrics-daemon-nthlf\" (UID: \"caba9223-8c40-4e7e-9740-98fcb97755b0\") " pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:58:32.042924 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.042845    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3b6e1a55-1afc-40a7-be4c-9f5810802717-original-pull-secret\") pod \"global-pull-secret-syncer-4k55t\" (UID: \"3b6e1a55-1afc-40a7-be4c-9f5810802717\") " pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:58:32.130616 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.130593    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:58:32.132949 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.132930    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 17:58:32.143904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.143888    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 17:58:32.154182 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.154161    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm8ps\" (UniqueName: \"kubernetes.io/projected/26478850-8053-48df-b800-bff475488a1e-kube-api-access-tm8ps\") pod \"network-check-target-n68jr\" (UID: \"26478850-8053-48df-b800-bff475488a1e\") " pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:58:32.224314 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.224294    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4k55t"
Jul 26 17:58:32.233391 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.233359    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9fbzz\""
Jul 26 17:58:32.238833 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.238815    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7vmj\""
Jul 26 17:58:32.240959 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.240834    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nthlf"
Jul 26 17:58:32.246728 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.246675    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:58:32.379025 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.378995    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4k55t"]
Jul 26 17:58:32.383432 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:32.383400    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6e1a55_1afc_40a7_be4c_9f5810802717.slice/crio-4ca7532fa46d9a68b96f120e7aac650a27fa3895f66a1d809fb6259d8a61105a WatchSource:0}: Error finding container 4ca7532fa46d9a68b96f120e7aac650a27fa3895f66a1d809fb6259d8a61105a: Status 404 returned error can't find the container with id 4ca7532fa46d9a68b96f120e7aac650a27fa3895f66a1d809fb6259d8a61105a
Jul 26 17:58:32.403427 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.403401    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nthlf"]
Jul 26 17:58:32.406408 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:32.406386    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaba9223_8c40_4e7e_9740_98fcb97755b0.slice/crio-3107d6da278bb4a7f5e5d1f646ce5cabeed2da37accc70d56b6d180e7a7f4ac9 WatchSource:0}: Error finding container 3107d6da278bb4a7f5e5d1f646ce5cabeed2da37accc70d56b6d180e7a7f4ac9: Status 404 returned error can't find the container with id 3107d6da278bb4a7f5e5d1f646ce5cabeed2da37accc70d56b6d180e7a7f4ac9
Jul 26 17:58:32.419385 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.419335    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n68jr"]
Jul 26 17:58:32.423108 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:32.423087    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26478850_8053_48df_b800_bff475488a1e.slice/crio-17f8ced25a31778d5c1d67ba64581d7aa35a19516589acc6e0cf943beb5dbe45 WatchSource:0}: Error finding container 17f8ced25a31778d5c1d67ba64581d7aa35a19516589acc6e0cf943beb5dbe45: Status 404 returned error can't find the container with id 17f8ced25a31778d5c1d67ba64581d7aa35a19516589acc6e0cf943beb5dbe45
Jul 26 17:58:32.690821 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.690729    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nthlf" event={"ID":"caba9223-8c40-4e7e-9740-98fcb97755b0","Type":"ContainerStarted","Data":"3107d6da278bb4a7f5e5d1f646ce5cabeed2da37accc70d56b6d180e7a7f4ac9"}
Jul 26 17:58:32.691828 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.691802    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4k55t" event={"ID":"3b6e1a55-1afc-40a7-be4c-9f5810802717","Type":"ContainerStarted","Data":"4ca7532fa46d9a68b96f120e7aac650a27fa3895f66a1d809fb6259d8a61105a"}
Jul 26 17:58:32.692938 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.692911    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n68jr" event={"ID":"26478850-8053-48df-b800-bff475488a1e","Type":"ContainerStarted","Data":"17f8ced25a31778d5c1d67ba64581d7aa35a19516589acc6e0cf943beb5dbe45"}
Jul 26 17:58:32.694801 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.694781    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" event={"ID":"5c67e896-4b1d-4684-9566-6811a410c830","Type":"ContainerStarted","Data":"e8840f4cfb6b34ef8a7a11e566d7a6117a34bc244545e0f37c62fd411e4e726c"}
Jul 26 17:58:32.694875 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.694809    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" event={"ID":"5c67e896-4b1d-4684-9566-6811a410c830","Type":"ContainerStarted","Data":"d276b90399649567dc0eeb64154291a1be59642cc068d3d65352ff87264ef30f"}
Jul 26 17:58:32.694875 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.694820    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" event={"ID":"5c67e896-4b1d-4684-9566-6811a410c830","Type":"ContainerStarted","Data":"e08f3203b78d4cbd4829800805355777683493ced4e8831b5071ab6c5a6d8e7f"}
Jul 26 17:58:32.715903 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:32.715848    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dbddcfdc6-jw66f" podStartSLOduration=4.330203953 podStartE2EDuration="7.715831733s" totalImagesPullingTime="3.38562778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:58:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:32.714020416 +0000 UTC m=+66.904050172" watchObservedRunningTime="2026-07-26 17:58:32.715831733 +0000 UTC m=+66.905861486"
Jul 26 17:58:33.702428 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:33.702106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nthlf" event={"ID":"caba9223-8c40-4e7e-9740-98fcb97755b0","Type":"ContainerStarted","Data":"69ac3825128ca54ef30ce69e4211eb0b20806eb19726ea883f0f980f0e0aa164"}
Jul 26 17:58:33.702428 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:33.702168    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nthlf" event={"ID":"caba9223-8c40-4e7e-9740-98fcb97755b0","Type":"ContainerStarted","Data":"9014adc67c6c1bba323025e1f898d57b03f381f3d1df42f192a068368bbcf546"}
Jul 26 17:58:36.065222 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:36.065188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:36.068070 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:36.068010    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6eff04b4-327d-4f73-b801-698564387ab4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kr997\" (UID: \"6eff04b4-327d-4f73-b801-698564387ab4\") " pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:36.108824 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:36.108792    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2d24l\""
Jul 26 17:58:36.116975 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:36.116952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kr997"
Jul 26 17:58:37.297917 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.297862    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nthlf" podStartSLOduration=69.329750343 podStartE2EDuration="1m10.297844717s" totalImagesPullingTime="968.094374ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:33.719117113 +0000 UTC m=+67.909146876" watchObservedRunningTime="2026-07-26 17:58:37.297844717 +0000 UTC m=+71.487874470"
Jul 26 17:58:37.298811 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.298787    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kr997"]
Jul 26 17:58:37.301992 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:37.301966    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eff04b4_327d_4f73_b801_698564387ab4.slice/crio-fa6dde73f05182afa458bf12d566a5214ed87bfd6763f318e517fe34576a791a WatchSource:0}: Error finding container fa6dde73f05182afa458bf12d566a5214ed87bfd6763f318e517fe34576a791a: Status 404 returned error can't find the container with id fa6dde73f05182afa458bf12d566a5214ed87bfd6763f318e517fe34576a791a
Jul 26 17:58:37.719518 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.719477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4k55t" event={"ID":"3b6e1a55-1afc-40a7-be4c-9f5810802717","Type":"ContainerStarted","Data":"f0803edfe94898e83f17536171d47b7993c1c22c598f8d311b25f903918d5ca0"}
Jul 26 17:58:37.721820 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.721773    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n68jr" event={"ID":"26478850-8053-48df-b800-bff475488a1e","Type":"ContainerStarted","Data":"842112fb451b9ed4be05aa342e8551d6a2a6a73d431eedaacf433d76ef320512"}
Jul 26 17:58:37.722419 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.722363    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:58:37.724267 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.723572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kr997" event={"ID":"6eff04b4-327d-4f73-b801-698564387ab4","Type":"ContainerStarted","Data":"fa6dde73f05182afa458bf12d566a5214ed87bfd6763f318e517fe34576a791a"}
Jul 26 17:58:37.737953 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:37.737900    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4k55t" podStartSLOduration=65.940431098 podStartE2EDuration="1m10.737880925s" totalImagesPullingTime="4.797449827s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:37.735934348 +0000 UTC m=+71.925964104" watchObservedRunningTime="2026-07-26 17:58:37.737880925 +0000 UTC m=+71.927910683"
Jul 26 17:58:42.743518 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:42.743410    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kr997" event={"ID":"6eff04b4-327d-4f73-b801-698564387ab4","Type":"ContainerStarted","Data":"2b02453572cbf3021ece019d23ca112f84494a50f79c6652dc30863dae3b51a8"}
Jul 26 17:58:43.831463 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.831414    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n68jr" podStartSLOduration=72.089373839 podStartE2EDuration="1m16.831396972s" totalImagesPullingTime="4.742023133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 17:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:37.754742818 +0000 UTC m=+71.944772589" watchObservedRunningTime="2026-07-26 17:58:43.831396972 +0000 UTC m=+78.021426726"
Jul 26 17:58:43.831930 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.831891    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c4cjk"]
Jul 26 17:58:43.842169 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.842139    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.845009 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.844944    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 17:58:43.845177 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.845122    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 17:58:43.845291 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.845186    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 17:58:43.846248 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.846227    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-47kbs\""
Jul 26 17:58:43.846248 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.846244    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 17:58:43.846404 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.846276    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 17:58:43.846404 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.846302    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 17:58:43.924688 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924644    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-wtmp\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.924890 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924712    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4rxm\" (UniqueName: \"kubernetes.io/projected/1620f426-1294-4f46-88f5-87cce4d3d73f-kube-api-access-k4rxm\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.924890 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924774    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-textfile\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.924890 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924814    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-tls\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.925057 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924897    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-root\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.925057 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924924    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-accelerators-collector-config\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.925057 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924948    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1620f426-1294-4f46-88f5-87cce4d3d73f-metrics-client-ca\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.925057 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.924983    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:43.925261 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:43.925078    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-sys\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026513 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026479    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-wtmp\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026689 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026528    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4rxm\" (UniqueName: \"kubernetes.io/projected/1620f426-1294-4f46-88f5-87cce4d3d73f-kube-api-access-k4rxm\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026689 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026561    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-textfile\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026689 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026587    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-tls\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026689 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026674    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-wtmp\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026782    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-root\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026825    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-accelerators-collector-config\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026860    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1620f426-1294-4f46-88f5-87cce4d3d73f-metrics-client-ca\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.026904 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026896    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.027098 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026924    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-textfile\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.027098 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-sys\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.027098 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.026928    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-sys\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.027098 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:44.027024    2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 26 17:58:44.027098 ip-10-0-133-34 kubenswrapper[2565]: E0726 17:58:44.027088    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-tls podName:1620f426-1294-4f46-88f5-87cce4d3d73f nodeName:}" failed. No retries permitted until 2026-07-26 17:58:44.527066631 +0000 UTC m=+78.717096364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-tls") pod "node-exporter-c4cjk" (UID: "1620f426-1294-4f46-88f5-87cce4d3d73f") : secret "node-exporter-tls" not found
Jul 26 17:58:44.027355 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.027278    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1620f426-1294-4f46-88f5-87cce4d3d73f-root\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.027525 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.027499    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1620f426-1294-4f46-88f5-87cce4d3d73f-metrics-client-ca\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.027609 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.027541    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-accelerators-collector-config\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.029916 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.029873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.037580 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.037555    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4rxm\" (UniqueName: \"kubernetes.io/projected/1620f426-1294-4f46-88f5-87cce4d3d73f-kube-api-access-k4rxm\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.531308 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.531262    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-tls\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.534029 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.534001    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1620f426-1294-4f46-88f5-87cce4d3d73f-node-exporter-tls\") pod \"node-exporter-c4cjk\" (UID: \"1620f426-1294-4f46-88f5-87cce4d3d73f\") " pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:44.753085 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:44.753059    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c4cjk"
Jul 26 17:58:45.450342 ip-10-0-133-34 kubenswrapper[2565]: W0726 17:58:45.450312    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1620f426_1294_4f46_88f5_87cce4d3d73f.slice/crio-b95cea6525b0a0738e92a04476eec348ac3bec484ba3981291d553edc4338f6b WatchSource:0}: Error finding container b95cea6525b0a0738e92a04476eec348ac3bec484ba3981291d553edc4338f6b: Status 404 returned error can't find the container with id b95cea6525b0a0738e92a04476eec348ac3bec484ba3981291d553edc4338f6b
Jul 26 17:58:45.756115 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:45.756081    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kr997" event={"ID":"6eff04b4-327d-4f73-b801-698564387ab4","Type":"ContainerStarted","Data":"e5b5945994a6655e2b868f3c4b0743aeed487cb89712729d60cb0bbbd3477a14"}
Jul 26 17:58:45.757412 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:45.757385    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c4cjk" event={"ID":"1620f426-1294-4f46-88f5-87cce4d3d73f","Type":"ContainerStarted","Data":"b95cea6525b0a0738e92a04476eec348ac3bec484ba3981291d553edc4338f6b"}
Jul 26 17:58:46.762532 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:46.762498    2565 generic.go:356] "Generic (PLEG): container finished" podID="1620f426-1294-4f46-88f5-87cce4d3d73f" containerID="3128b1356f2fb7c17fa43969ce3c329fa5cc99ccda26f2acfad7de172da47194" exitCode=0
Jul 26 17:58:46.763154 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:46.762553    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c4cjk" event={"ID":"1620f426-1294-4f46-88f5-87cce4d3d73f","Type":"ContainerDied","Data":"3128b1356f2fb7c17fa43969ce3c329fa5cc99ccda26f2acfad7de172da47194"}
Jul 26 17:58:47.769765 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:47.769729    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c4cjk" event={"ID":"1620f426-1294-4f46-88f5-87cce4d3d73f","Type":"ContainerStarted","Data":"92aa1edcaf684b9641696db1d7221403d9f289b1d45c397fb0ad8994653a2b92"}
Jul 26 17:58:47.770212 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:47.769789    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c4cjk" event={"ID":"1620f426-1294-4f46-88f5-87cce4d3d73f","Type":"ContainerStarted","Data":"b70f2c504c1ddf968132d7286859d20446da43267f3b0e53e316ca8fdcc59035"}
Jul 26 17:58:47.792742 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:47.792669    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c4cjk" podStartSLOduration=3.955797866 podStartE2EDuration="4.792647475s" totalImagesPullingTime="758.530066ms" totalInitContainerRuntime="78.319543ms" isStatefulPod=true podCreationTimestamp="2026-07-26 17:58:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:47.791066434 +0000 UTC m=+81.981096191" watchObservedRunningTime="2026-07-26 17:58:47.792647475 +0000 UTC m=+81.982677230"
Jul 26 17:58:49.778485 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:49.778454    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kr997" event={"ID":"6eff04b4-327d-4f73-b801-698564387ab4","Type":"ContainerStarted","Data":"999a25cd65212329510eabf690cdf92ee6b7e6f1ad31abd7c851ebffba9cecbb"}
Jul 26 17:58:49.800903 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:58:49.800851    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kr997" podStartSLOduration=33.982740627 podStartE2EDuration="45.80083543s" totalImagesPullingTime="11.818094803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 17:58:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 17:58:49.796233317 +0000 UTC m=+83.986263075" watchObservedRunningTime="2026-07-26 17:58:49.80083543 +0000 UTC m=+83.990865184"
Jul 26 17:59:09.733250 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:59:09.733221    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n68jr"
Jul 26 17:59:23.512230 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:59:23.512199    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c4cjk_1620f426-1294-4f46-88f5-87cce4d3d73f/init-textfile/0.log"
Jul 26 17:59:23.712710 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:59:23.712665    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c4cjk_1620f426-1294-4f46-88f5-87cce4d3d73f/node-exporter/0.log"
Jul 26 17:59:23.912234 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:59:23.912120    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c4cjk_1620f426-1294-4f46-88f5-87cce4d3d73f/kube-rbac-proxy/0.log"
Jul 26 17:59:29.912394 ip-10-0-133-34 kubenswrapper[2565]: I0726 17:59:29.912366    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6jtsk_23eee869-252e-495d-9a41-efaf13e2e3e6/serve-healthcheck-canary/0.log"
Jul 26 18:02:23.970455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:23.970422    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"]
Jul 26 18:02:23.973625 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:23.973604    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:23.976217 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:23.976193    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-78bch\""
Jul 26 18:02:23.976392 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:23.976377    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 26 18:02:23.977372 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:23.977359    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 26 18:02:23.982388 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:23.982368    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"]
Jul 26 18:02:24.136547 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.136510    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.136720 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.136631    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.136720 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.136664    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbpl9\" (UniqueName: \"kubernetes.io/projected/bb396d7c-b61c-4692-860f-00f78d87713b-kube-api-access-fbpl9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.237848 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.237769    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.237848 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.237836    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.237979 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.237855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbpl9\" (UniqueName: \"kubernetes.io/projected/bb396d7c-b61c-4692-860f-00f78d87713b-kube-api-access-fbpl9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.238149 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.238129    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.238185 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.238163    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.246080 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.246050    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbpl9\" (UniqueName: \"kubernetes.io/projected/bb396d7c-b61c-4692-860f-00f78d87713b-kube-api-access-fbpl9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.281982 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.281938    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:24.410630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.410600    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"]
Jul 26 18:02:24.415180 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:02:24.415146    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb396d7c_b61c_4692_860f_00f78d87713b.slice/crio-0d6e83e7ea0553e1d17e3995ae3dc61caa1055136856065537dcc7ceefcc9dc8 WatchSource:0}: Error finding container 0d6e83e7ea0553e1d17e3995ae3dc61caa1055136856065537dcc7ceefcc9dc8: Status 404 returned error can't find the container with id 0d6e83e7ea0553e1d17e3995ae3dc61caa1055136856065537dcc7ceefcc9dc8
Jul 26 18:02:24.459542 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:24.459511    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" event={"ID":"bb396d7c-b61c-4692-860f-00f78d87713b","Type":"ContainerStarted","Data":"0d6e83e7ea0553e1d17e3995ae3dc61caa1055136856065537dcc7ceefcc9dc8"}
Jul 26 18:02:26.127977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:26.127951    2565 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 18:02:30.483251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:30.483217    2565 generic.go:356] "Generic (PLEG): container finished" podID="bb396d7c-b61c-4692-860f-00f78d87713b" containerID="ba7b506e571f6f31e8ced07f85e64b961cc9da05b9bce3816fbaaa0477f59d76" exitCode=0
Jul 26 18:02:30.483669 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:30.483289    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" event={"ID":"bb396d7c-b61c-4692-860f-00f78d87713b","Type":"ContainerDied","Data":"ba7b506e571f6f31e8ced07f85e64b961cc9da05b9bce3816fbaaa0477f59d76"}
Jul 26 18:02:30.484253 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:30.484235    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:02:33.496243 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:33.496210    2565 generic.go:356] "Generic (PLEG): container finished" podID="bb396d7c-b61c-4692-860f-00f78d87713b" containerID="4ee9758c529b5d1a37478ad94d354fb6a34c49cc7a642dc65aa81c5dc07c1636" exitCode=0
Jul 26 18:02:33.496638 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:33.496277    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" event={"ID":"bb396d7c-b61c-4692-860f-00f78d87713b","Type":"ContainerDied","Data":"4ee9758c529b5d1a37478ad94d354fb6a34c49cc7a642dc65aa81c5dc07c1636"}
Jul 26 18:02:40.525151 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:40.525109    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" event={"ID":"bb396d7c-b61c-4692-860f-00f78d87713b","Type":"ContainerStarted","Data":"03e9e8a2edb5a79529d9d639e46a297ce0a847c97d7c47e1534d814019172446"}
Jul 26 18:02:40.542078 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:40.542029    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" podStartSLOduration=2.78003996 podStartE2EDuration="17.542012625s" totalImagesPullingTime="14.761972665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:02:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:02:40.541220851 +0000 UTC m=+314.731250605" watchObservedRunningTime="2026-07-26 18:02:40.542012625 +0000 UTC m=+314.732042378"
Jul 26 18:02:41.530693 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:41.530659    2565 generic.go:356] "Generic (PLEG): container finished" podID="bb396d7c-b61c-4692-860f-00f78d87713b" containerID="03e9e8a2edb5a79529d9d639e46a297ce0a847c97d7c47e1534d814019172446" exitCode=0
Jul 26 18:02:41.531081 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:41.530751    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" event={"ID":"bb396d7c-b61c-4692-860f-00f78d87713b","Type":"ContainerDied","Data":"03e9e8a2edb5a79529d9d639e46a297ce0a847c97d7c47e1534d814019172446"}
Jul 26 18:02:42.661563 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.661542    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:42.674176 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.674152    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbpl9\" (UniqueName: \"kubernetes.io/projected/bb396d7c-b61c-4692-860f-00f78d87713b-kube-api-access-fbpl9\") pod \"bb396d7c-b61c-4692-860f-00f78d87713b\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") "
Jul 26 18:02:42.674278 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.674205    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-util\") pod \"bb396d7c-b61c-4692-860f-00f78d87713b\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") "
Jul 26 18:02:42.676241 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.676215    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb396d7c-b61c-4692-860f-00f78d87713b-kube-api-access-fbpl9" (OuterVolumeSpecName: "kube-api-access-fbpl9") pod "bb396d7c-b61c-4692-860f-00f78d87713b" (UID: "bb396d7c-b61c-4692-860f-00f78d87713b"). InnerVolumeSpecName "kube-api-access-fbpl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:02:42.679185 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.679129    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-util" (OuterVolumeSpecName: "util") pod "bb396d7c-b61c-4692-860f-00f78d87713b" (UID: "bb396d7c-b61c-4692-860f-00f78d87713b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:02:42.774528 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.774495    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-bundle\") pod \"bb396d7c-b61c-4692-860f-00f78d87713b\" (UID: \"bb396d7c-b61c-4692-860f-00f78d87713b\") "
Jul 26 18:02:42.774694 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.774637    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:02:42.774694 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.774649    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbpl9\" (UniqueName: \"kubernetes.io/projected/bb396d7c-b61c-4692-860f-00f78d87713b-kube-api-access-fbpl9\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:02:42.775154 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.775129    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-bundle" (OuterVolumeSpecName: "bundle") pod "bb396d7c-b61c-4692-860f-00f78d87713b" (UID: "bb396d7c-b61c-4692-860f-00f78d87713b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:02:42.874965 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:42.874891    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb396d7c-b61c-4692-860f-00f78d87713b-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:02:43.541789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:43.541755    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn" event={"ID":"bb396d7c-b61c-4692-860f-00f78d87713b","Type":"ContainerDied","Data":"0d6e83e7ea0553e1d17e3995ae3dc61caa1055136856065537dcc7ceefcc9dc8"}
Jul 26 18:02:43.541789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:43.541780    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pzrfn"
Jul 26 18:02:43.542001 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:43.541803    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d6e83e7ea0553e1d17e3995ae3dc61caa1055136856065537dcc7ceefcc9dc8"
Jul 26 18:02:51.320230 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320199    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"]
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320411    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="util"
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320424    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="util"
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320435    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="pull"
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320441    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="pull"
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320449    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="extract"
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320455    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="extract"
Jul 26 18:02:51.320689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.320497    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb396d7c-b61c-4692-860f-00f78d87713b" containerName="extract"
Jul 26 18:02:51.323321 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.323299    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.326379 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.326357    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 26 18:02:51.326679 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.326662    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 26 18:02:51.327989 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.327976    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-78bch\""
Jul 26 18:02:51.329478 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.329386    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.329478 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.329424    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.329598 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.329532    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7jpx\" (UniqueName: \"kubernetes.io/projected/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-kube-api-access-m7jpx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.344671 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.344647    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"]
Jul 26 18:02:51.429913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.429876    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7jpx\" (UniqueName: \"kubernetes.io/projected/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-kube-api-access-m7jpx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.430112 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.429936    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.430112 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.429965    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.430386 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.430363    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.430386 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.430373    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.439477 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.439447    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7jpx\" (UniqueName: \"kubernetes.io/projected/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-kube-api-access-m7jpx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.631792 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.631679    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:51.677005 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.676976    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k5pdw"]
Jul 26 18:02:51.680575 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.680506    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.683295 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.683265    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zf8zc\""
Jul 26 18:02:51.683421 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.683345    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 26 18:02:51.683477 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.683265    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 26 18:02:51.701675 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.701640    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k5pdw"]
Jul 26 18:02:51.731678 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.731513    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn6sd\" (UniqueName: \"kubernetes.io/projected/b42e1033-911c-4dca-8edf-5a7ba8a9d017-kube-api-access-dn6sd\") pod \"cert-manager-webhook-6486ffd997-k5pdw\" (UID: \"b42e1033-911c-4dca-8edf-5a7ba8a9d017\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.731678 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.731573    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b42e1033-911c-4dca-8edf-5a7ba8a9d017-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k5pdw\" (UID: \"b42e1033-911c-4dca-8edf-5a7ba8a9d017\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.766560 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.766537    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"]
Jul 26 18:02:51.768714 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:02:51.768672    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e25a11e_85a6_4dee_b737_44ff2b6b69d9.slice/crio-14f8209484d7afeda2354ac870b6a63bb3d83691259ddec890701c9a175cedec WatchSource:0}: Error finding container 14f8209484d7afeda2354ac870b6a63bb3d83691259ddec890701c9a175cedec: Status 404 returned error can't find the container with id 14f8209484d7afeda2354ac870b6a63bb3d83691259ddec890701c9a175cedec
Jul 26 18:02:51.832261 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.832237    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b42e1033-911c-4dca-8edf-5a7ba8a9d017-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k5pdw\" (UID: \"b42e1033-911c-4dca-8edf-5a7ba8a9d017\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.832367 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.832331    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn6sd\" (UniqueName: \"kubernetes.io/projected/b42e1033-911c-4dca-8edf-5a7ba8a9d017-kube-api-access-dn6sd\") pod \"cert-manager-webhook-6486ffd997-k5pdw\" (UID: \"b42e1033-911c-4dca-8edf-5a7ba8a9d017\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.847549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.847525    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b42e1033-911c-4dca-8edf-5a7ba8a9d017-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-k5pdw\" (UID: \"b42e1033-911c-4dca-8edf-5a7ba8a9d017\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.847676 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.847636    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn6sd\" (UniqueName: \"kubernetes.io/projected/b42e1033-911c-4dca-8edf-5a7ba8a9d017-kube-api-access-dn6sd\") pod \"cert-manager-webhook-6486ffd997-k5pdw\" (UID: \"b42e1033-911c-4dca-8edf-5a7ba8a9d017\") " pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:51.991935 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:51.991908    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:52.122323 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:52.122299    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-k5pdw"]
Jul 26 18:02:52.124628 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:02:52.124599    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb42e1033_911c_4dca_8edf_5a7ba8a9d017.slice/crio-a70fb11e26d67ede3b5447c5af319596afed6da7918cc1376eabcc0e8e5199fc WatchSource:0}: Error finding container a70fb11e26d67ede3b5447c5af319596afed6da7918cc1376eabcc0e8e5199fc: Status 404 returned error can't find the container with id a70fb11e26d67ede3b5447c5af319596afed6da7918cc1376eabcc0e8e5199fc
Jul 26 18:02:52.573310 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:52.573278    2565 generic.go:356] "Generic (PLEG): container finished" podID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerID="4ab730076770279e18ca18021f415d7f5ec99a4d1af2230e48061df5bf398d51" exitCode=0
Jul 26 18:02:52.573730 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:52.573364    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65" event={"ID":"7e25a11e-85a6-4dee-b737-44ff2b6b69d9","Type":"ContainerDied","Data":"4ab730076770279e18ca18021f415d7f5ec99a4d1af2230e48061df5bf398d51"}
Jul 26 18:02:52.573730 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:52.573415    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65" event={"ID":"7e25a11e-85a6-4dee-b737-44ff2b6b69d9","Type":"ContainerStarted","Data":"14f8209484d7afeda2354ac870b6a63bb3d83691259ddec890701c9a175cedec"}
Jul 26 18:02:52.574598 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:52.574568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw" event={"ID":"b42e1033-911c-4dca-8edf-5a7ba8a9d017","Type":"ContainerStarted","Data":"a70fb11e26d67ede3b5447c5af319596afed6da7918cc1376eabcc0e8e5199fc"}
Jul 26 18:02:55.588203 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:55.588167    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw" event={"ID":"b42e1033-911c-4dca-8edf-5a7ba8a9d017","Type":"ContainerStarted","Data":"b432fa345a6c21444719d971477f83aca5e4e0b32fafad98b76953c5f27b4638"}
Jul 26 18:02:55.588598 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:55.588328    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:02:55.589989 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:55.589969    2565 generic.go:356] "Generic (PLEG): container finished" podID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerID="af2880d3bb1f4552a9849b649ee2a601711d7f2ac04cc1e1e6e0785f271f275b" exitCode=0
Jul 26 18:02:55.590088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:55.590006    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65" event={"ID":"7e25a11e-85a6-4dee-b737-44ff2b6b69d9","Type":"ContainerDied","Data":"af2880d3bb1f4552a9849b649ee2a601711d7f2ac04cc1e1e6e0785f271f275b"}
Jul 26 18:02:55.606355 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:55.606315    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw" podStartSLOduration=1.347127419 podStartE2EDuration="4.606299612s" totalImagesPullingTime="3.259172193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:02:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:02:55.60539231 +0000 UTC m=+329.795422083" watchObservedRunningTime="2026-07-26 18:02:55.606299612 +0000 UTC m=+329.796329365"
Jul 26 18:02:56.595427 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:56.595396    2565 generic.go:356] "Generic (PLEG): container finished" podID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerID="6f936e8c17073984e2fe5b6ace962ae21f7fb02d82b51259d28141bb24def5a9" exitCode=0
Jul 26 18:02:56.595868 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:56.595477    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65" event={"ID":"7e25a11e-85a6-4dee-b737-44ff2b6b69d9","Type":"ContainerDied","Data":"6f936e8c17073984e2fe5b6ace962ae21f7fb02d82b51259d28141bb24def5a9"}
Jul 26 18:02:57.719757 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.719728    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:02:57.779346 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.779312    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7jpx\" (UniqueName: \"kubernetes.io/projected/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-kube-api-access-m7jpx\") pod \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") "
Jul 26 18:02:57.779530 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.779359    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-bundle\") pod \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") "
Jul 26 18:02:57.779530 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.779407    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-util\") pod \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\" (UID: \"7e25a11e-85a6-4dee-b737-44ff2b6b69d9\") "
Jul 26 18:02:57.779809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.779774    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-bundle" (OuterVolumeSpecName: "bundle") pod "7e25a11e-85a6-4dee-b737-44ff2b6b69d9" (UID: "7e25a11e-85a6-4dee-b737-44ff2b6b69d9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:02:57.781421 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.781392    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-kube-api-access-m7jpx" (OuterVolumeSpecName: "kube-api-access-m7jpx") pod "7e25a11e-85a6-4dee-b737-44ff2b6b69d9" (UID: "7e25a11e-85a6-4dee-b737-44ff2b6b69d9"). InnerVolumeSpecName "kube-api-access-m7jpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:02:57.784490 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.784467    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-util" (OuterVolumeSpecName: "util") pod "7e25a11e-85a6-4dee-b737-44ff2b6b69d9" (UID: "7e25a11e-85a6-4dee-b737-44ff2b6b69d9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:02:57.880400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.880322    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:02:57.880400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.880347    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:02:57.880400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:57.880358    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7jpx\" (UniqueName: \"kubernetes.io/projected/7e25a11e-85a6-4dee-b737-44ff2b6b69d9-kube-api-access-m7jpx\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:02:58.604145 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:58.604057    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65" event={"ID":"7e25a11e-85a6-4dee-b737-44ff2b6b69d9","Type":"ContainerDied","Data":"14f8209484d7afeda2354ac870b6a63bb3d83691259ddec890701c9a175cedec"}
Jul 26 18:02:58.604145 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:58.604106    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14f8209484d7afeda2354ac870b6a63bb3d83691259ddec890701c9a175cedec"
Jul 26 18:02:58.604145 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:02:58.604067    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fctz65"
Jul 26 18:03:01.598213 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:01.598184    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-k5pdw"
Jul 26 18:03:04.018026 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.017989    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hvh9d"]
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018238    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="util"
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018249    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="util"
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018255    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="pull"
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018261    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="pull"
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018273    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="extract"
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018278    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="extract"
Jul 26 18:03:04.018435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.018314    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e25a11e-85a6-4dee-b737-44ff2b6b69d9" containerName="extract"
Jul 26 18:03:04.023926 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.023906    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.026981 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.026957    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f6xv2\""
Jul 26 18:03:04.040809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.040784    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hvh9d"]
Jul 26 18:03:04.129476 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.129436    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4bwv\" (UniqueName: \"kubernetes.io/projected/e94eb31b-8472-4242-8a7b-f0b6e29120eb-kube-api-access-q4bwv\") pod \"cert-manager-55b4d954d5-hvh9d\" (UID: \"e94eb31b-8472-4242-8a7b-f0b6e29120eb\") " pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.129476 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.129472    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e94eb31b-8472-4242-8a7b-f0b6e29120eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-hvh9d\" (UID: \"e94eb31b-8472-4242-8a7b-f0b6e29120eb\") " pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.230230 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.230189    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4bwv\" (UniqueName: \"kubernetes.io/projected/e94eb31b-8472-4242-8a7b-f0b6e29120eb-kube-api-access-q4bwv\") pod \"cert-manager-55b4d954d5-hvh9d\" (UID: \"e94eb31b-8472-4242-8a7b-f0b6e29120eb\") " pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.230230 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.230233    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e94eb31b-8472-4242-8a7b-f0b6e29120eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-hvh9d\" (UID: \"e94eb31b-8472-4242-8a7b-f0b6e29120eb\") " pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.237933 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.237906    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e94eb31b-8472-4242-8a7b-f0b6e29120eb-bound-sa-token\") pod \"cert-manager-55b4d954d5-hvh9d\" (UID: \"e94eb31b-8472-4242-8a7b-f0b6e29120eb\") " pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.238126 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.238104    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4bwv\" (UniqueName: \"kubernetes.io/projected/e94eb31b-8472-4242-8a7b-f0b6e29120eb-kube-api-access-q4bwv\") pod \"cert-manager-55b4d954d5-hvh9d\" (UID: \"e94eb31b-8472-4242-8a7b-f0b6e29120eb\") " pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.332598 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.332512    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hvh9d"
Jul 26 18:03:04.499176 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.499144    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hvh9d"]
Jul 26 18:03:04.502032 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:03:04.502010    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode94eb31b_8472_4242_8a7b_f0b6e29120eb.slice/crio-7a6b20dc90745126883efd25b9e0db76d382c4f8094c307879e99da4dd9ae6cf WatchSource:0}: Error finding container 7a6b20dc90745126883efd25b9e0db76d382c4f8094c307879e99da4dd9ae6cf: Status 404 returned error can't find the container with id 7a6b20dc90745126883efd25b9e0db76d382c4f8094c307879e99da4dd9ae6cf
Jul 26 18:03:04.626053 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.626019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hvh9d" event={"ID":"e94eb31b-8472-4242-8a7b-f0b6e29120eb","Type":"ContainerStarted","Data":"c9ff74500da16ab8d6baa2bb418ca7e487052cd6b07d2423fd4ba7e1789000c4"}
Jul 26 18:03:04.626182 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:04.626066    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hvh9d" event={"ID":"e94eb31b-8472-4242-8a7b-f0b6e29120eb","Type":"ContainerStarted","Data":"7a6b20dc90745126883efd25b9e0db76d382c4f8094c307879e99da4dd9ae6cf"}
Jul 26 18:03:05.645994 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:05.645944    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hvh9d" podStartSLOduration=1.6459280330000001 podStartE2EDuration="1.645928033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:03:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:03:05.644733586 +0000 UTC m=+339.834763340" watchObservedRunningTime="2026-07-26 18:03:05.645928033 +0000 UTC m=+339.835957787"
Jul 26 18:03:17.067638 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.067599    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"]
Jul 26 18:03:17.073364 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.073346    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.076710 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.076671    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 26 18:03:17.076816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.076770    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 26 18:03:17.077737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.077722    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-78bch\""
Jul 26 18:03:17.096680 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.096660    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"]
Jul 26 18:03:17.116109 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.116088    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.116214 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.116119    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.116214 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.116143    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qfwx\" (UniqueName: \"kubernetes.io/projected/168c2b42-5094-4cd4-86c9-19182db6cdf1-kube-api-access-6qfwx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.217229 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.217174    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.217229 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.217229    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.217497 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.217332    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qfwx\" (UniqueName: \"kubernetes.io/projected/168c2b42-5094-4cd4-86c9-19182db6cdf1-kube-api-access-6qfwx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.217637 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.217615    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.217737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.217642    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.228153 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.228131    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qfwx\" (UniqueName: \"kubernetes.io/projected/168c2b42-5094-4cd4-86c9-19182db6cdf1-kube-api-access-6qfwx\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.382227 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.382138    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:17.504751 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.504729    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"]
Jul 26 18:03:17.507104 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:03:17.507073    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168c2b42_5094_4cd4_86c9_19182db6cdf1.slice/crio-eec80886033ebf7648e87fddda9959f94773644fa75cc234fa5ee80c1182072e WatchSource:0}: Error finding container eec80886033ebf7648e87fddda9959f94773644fa75cc234fa5ee80c1182072e: Status 404 returned error can't find the container with id eec80886033ebf7648e87fddda9959f94773644fa75cc234fa5ee80c1182072e
Jul 26 18:03:17.675507 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.675434    2565 generic.go:356] "Generic (PLEG): container finished" podID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerID="f93ce7a8bf96a01dc987ac0de2295bb10e1b6a1016e05fc7df39b777ab7d8c3c" exitCode=0
Jul 26 18:03:17.675507 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.675497    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59" event={"ID":"168c2b42-5094-4cd4-86c9-19182db6cdf1","Type":"ContainerDied","Data":"f93ce7a8bf96a01dc987ac0de2295bb10e1b6a1016e05fc7df39b777ab7d8c3c"}
Jul 26 18:03:17.675772 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:17.675538    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59" event={"ID":"168c2b42-5094-4cd4-86c9-19182db6cdf1","Type":"ContainerStarted","Data":"eec80886033ebf7648e87fddda9959f94773644fa75cc234fa5ee80c1182072e"}
Jul 26 18:03:18.680569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:18.680477    2565 generic.go:356] "Generic (PLEG): container finished" podID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerID="625a5e1f3ff6cb139a692454371f73e1b7d7d463a9f4d4ea32881990105596e3" exitCode=0
Jul 26 18:03:18.680569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:18.680522    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59" event={"ID":"168c2b42-5094-4cd4-86c9-19182db6cdf1","Type":"ContainerDied","Data":"625a5e1f3ff6cb139a692454371f73e1b7d7d463a9f4d4ea32881990105596e3"}
Jul 26 18:03:19.686301 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:19.686270    2565 generic.go:356] "Generic (PLEG): container finished" podID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerID="c2cef4aaff2e27e68cbf3c1c8704da6c1f1592b5ca322cded32312c617fdc210" exitCode=0
Jul 26 18:03:19.686714 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:19.686328    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59" event={"ID":"168c2b42-5094-4cd4-86c9-19182db6cdf1","Type":"ContainerDied","Data":"c2cef4aaff2e27e68cbf3c1c8704da6c1f1592b5ca322cded32312c617fdc210"}
Jul 26 18:03:20.816098 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.816069    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:20.942972 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.942884    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qfwx\" (UniqueName: \"kubernetes.io/projected/168c2b42-5094-4cd4-86c9-19182db6cdf1-kube-api-access-6qfwx\") pod \"168c2b42-5094-4cd4-86c9-19182db6cdf1\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") "
Jul 26 18:03:20.942972 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.942934    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-bundle\") pod \"168c2b42-5094-4cd4-86c9-19182db6cdf1\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") "
Jul 26 18:03:20.943156 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.942977    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-util\") pod \"168c2b42-5094-4cd4-86c9-19182db6cdf1\" (UID: \"168c2b42-5094-4cd4-86c9-19182db6cdf1\") "
Jul 26 18:03:20.943880 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.943854    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-bundle" (OuterVolumeSpecName: "bundle") pod "168c2b42-5094-4cd4-86c9-19182db6cdf1" (UID: "168c2b42-5094-4cd4-86c9-19182db6cdf1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:03:20.945002 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.944976    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/168c2b42-5094-4cd4-86c9-19182db6cdf1-kube-api-access-6qfwx" (OuterVolumeSpecName: "kube-api-access-6qfwx") pod "168c2b42-5094-4cd4-86c9-19182db6cdf1" (UID: "168c2b42-5094-4cd4-86c9-19182db6cdf1"). InnerVolumeSpecName "kube-api-access-6qfwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:03:20.949467 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:20.949446    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-util" (OuterVolumeSpecName: "util") pod "168c2b42-5094-4cd4-86c9-19182db6cdf1" (UID: "168c2b42-5094-4cd4-86c9-19182db6cdf1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:03:21.043958 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:21.043922    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qfwx\" (UniqueName: \"kubernetes.io/projected/168c2b42-5094-4cd4-86c9-19182db6cdf1-kube-api-access-6qfwx\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:03:21.043958 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:21.043953    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:03:21.043958 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:21.043963    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/168c2b42-5094-4cd4-86c9-19182db6cdf1-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:03:21.694420 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:21.694394    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59"
Jul 26 18:03:21.694420 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:21.694403    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affhv59" event={"ID":"168c2b42-5094-4cd4-86c9-19182db6cdf1","Type":"ContainerDied","Data":"eec80886033ebf7648e87fddda9959f94773644fa75cc234fa5ee80c1182072e"}
Jul 26 18:03:21.694620 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:21.694442    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eec80886033ebf7648e87fddda9959f94773644fa75cc234fa5ee80c1182072e"
Jul 26 18:03:28.364619 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364587    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"]
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364829    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="pull"
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364839    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="pull"
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364849    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="util"
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364855    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="util"
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364861    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="extract"
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364866    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="extract"
Jul 26 18:03:28.365084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.364907    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="168c2b42-5094-4cd4-86c9-19182db6cdf1" containerName="extract"
Jul 26 18:03:28.368846 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.368829    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.373926 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.373901    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 18:03:28.374059 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.373967    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 18:03:28.374386 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.374363    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sx8j6\""
Jul 26 18:03:28.374386 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.374363    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 18:03:28.374688 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.374664    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 18:03:28.386406 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.386383    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"]
Jul 26 18:03:28.501341 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.501305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71a8f8ca-4ce9-43d1-810b-21921235b7da-tmp\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.501341 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.501349    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71a8f8ca-4ce9-43d1-810b-21921235b7da-manifests\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.501574 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.501371    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71a8f8ca-4ce9-43d1-810b-21921235b7da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.501574 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.501400    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7wt\" (UniqueName: \"kubernetes.io/projected/71a8f8ca-4ce9-43d1-810b-21921235b7da-kube-api-access-ds7wt\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.501574 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.501428    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71a8f8ca-4ce9-43d1-810b-21921235b7da-webhook-cert\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.602552 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.602521    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71a8f8ca-4ce9-43d1-810b-21921235b7da-webhook-cert\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.602633 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.602574    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71a8f8ca-4ce9-43d1-810b-21921235b7da-tmp\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.602633 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.602600    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71a8f8ca-4ce9-43d1-810b-21921235b7da-manifests\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.602633 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.602618    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71a8f8ca-4ce9-43d1-810b-21921235b7da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.602807 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.602636    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7wt\" (UniqueName: \"kubernetes.io/projected/71a8f8ca-4ce9-43d1-810b-21921235b7da-kube-api-access-ds7wt\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.603042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.603018    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/71a8f8ca-4ce9-43d1-810b-21921235b7da-manifests\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.605006 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.604981    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71a8f8ca-4ce9-43d1-810b-21921235b7da-tmp\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.605115 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.605083    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71a8f8ca-4ce9-43d1-810b-21921235b7da-webhook-cert\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.605237 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.605221    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/71a8f8ca-4ce9-43d1-810b-21921235b7da-apiservice-cert\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.616265 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.616215    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7wt\" (UniqueName: \"kubernetes.io/projected/71a8f8ca-4ce9-43d1-810b-21921235b7da-kube-api-access-ds7wt\") pod \"opendatahub-operator-controller-manager-69cc94bb6c-xdsbf\" (UID: \"71a8f8ca-4ce9-43d1-810b-21921235b7da\") " pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.678426 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.678378    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:28.810791 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:28.810761    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"]
Jul 26 18:03:28.813846 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:03:28.813822    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71a8f8ca_4ce9_43d1_810b_21921235b7da.slice/crio-3d1ee5f9fa43c1f58791932343ccf26e859e0b15d4657da433d4a639c0bf1916 WatchSource:0}: Error finding container 3d1ee5f9fa43c1f58791932343ccf26e859e0b15d4657da433d4a639c0bf1916: Status 404 returned error can't find the container with id 3d1ee5f9fa43c1f58791932343ccf26e859e0b15d4657da433d4a639c0bf1916
Jul 26 18:03:29.726190 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:29.726136    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf" event={"ID":"71a8f8ca-4ce9-43d1-810b-21921235b7da","Type":"ContainerStarted","Data":"3d1ee5f9fa43c1f58791932343ccf26e859e0b15d4657da433d4a639c0bf1916"}
Jul 26 18:03:32.739324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:32.739281    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf" event={"ID":"71a8f8ca-4ce9-43d1-810b-21921235b7da","Type":"ContainerStarted","Data":"4bf94a6c3c69c8c07502438e2a622fd6adcb541cdf5bfeef0742ecacf6e564ef"}
Jul 26 18:03:33.746263 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:33.746232    2565 generic.go:356] "Generic (PLEG): container finished" podID="71a8f8ca-4ce9-43d1-810b-21921235b7da" containerID="4bf94a6c3c69c8c07502438e2a622fd6adcb541cdf5bfeef0742ecacf6e564ef" exitCode=0
Jul 26 18:03:33.746779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:33.746316    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf" event={"ID":"71a8f8ca-4ce9-43d1-810b-21921235b7da","Type":"ContainerDied","Data":"4bf94a6c3c69c8c07502438e2a622fd6adcb541cdf5bfeef0742ecacf6e564ef"}
Jul 26 18:03:34.752381 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:34.752338    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf" event={"ID":"71a8f8ca-4ce9-43d1-810b-21921235b7da","Type":"ContainerStarted","Data":"9aad1148d2f2a062e6f186cfe9bd1288384752ae5254a791d674b34d124f0557"}
Jul 26 18:03:34.752873 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:34.752480    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:34.774899 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:34.774852    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf" podStartSLOduration=2.443533339 podStartE2EDuration="6.774832771s" totalImagesPullingTime="3.318018321s" totalInitContainerRuntime="1.013281111s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:03:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:03:34.774290901 +0000 UTC m=+368.964320675" watchObservedRunningTime="2026-07-26 18:03:34.774832771 +0000 UTC m=+368.964862525"
Jul 26 18:03:45.759435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:45.759404    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-69cc94bb6c-xdsbf"
Jul 26 18:03:48.356127 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.356090    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"]
Jul 26 18:03:48.369182 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.369152    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.371848 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.371830    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 18:03:48.371971 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.371831    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 18:03:48.373047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.373025    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 18:03:48.373178 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.373078    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-562cq\""
Jul 26 18:03:48.373178 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.373081    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 18:03:48.375045 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.375020    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"]
Jul 26 18:03:48.452584 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.452549    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-tls-certs\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.452778 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.452606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjjhq\" (UniqueName: \"kubernetes.io/projected/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-kube-api-access-kjjhq\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.452778 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.452715    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-tmp\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.553363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.553333    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjjhq\" (UniqueName: \"kubernetes.io/projected/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-kube-api-access-kjjhq\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.553503 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.553386    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-tmp\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.553503 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.553410    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-tls-certs\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.555633 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.555607    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-tmp\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.555755 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.555738    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-tls-certs\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.566883 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.566861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjjhq\" (UniqueName: \"kubernetes.io/projected/ed75bbf9-d69e-43ce-b29b-81acaa4e5213-kube-api-access-kjjhq\") pod \"kube-auth-proxy-679f49c568-4rwtq\" (UID: \"ed75bbf9-d69e-43ce-b29b-81acaa4e5213\") " pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.679590 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.679511    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"
Jul 26 18:03:48.811392 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:48.811369    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-679f49c568-4rwtq"]
Jul 26 18:03:48.813513 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:03:48.813493    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded75bbf9_d69e_43ce_b29b_81acaa4e5213.slice/crio-7276a3a4e82f7eb86eaf636e917a9cf9e7ecb956e521229cb282d8cab884b719 WatchSource:0}: Error finding container 7276a3a4e82f7eb86eaf636e917a9cf9e7ecb956e521229cb282d8cab884b719: Status 404 returned error can't find the container with id 7276a3a4e82f7eb86eaf636e917a9cf9e7ecb956e521229cb282d8cab884b719
Jul 26 18:03:49.809209 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:49.809177    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq" event={"ID":"ed75bbf9-d69e-43ce-b29b-81acaa4e5213","Type":"ContainerStarted","Data":"7276a3a4e82f7eb86eaf636e917a9cf9e7ecb956e521229cb282d8cab884b719"}
Jul 26 18:03:51.818378 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:51.818339    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq" event={"ID":"ed75bbf9-d69e-43ce-b29b-81acaa4e5213","Type":"ContainerStarted","Data":"57b95f43ea5a77e6452d10e53bc07883a1a3e49afd738e2d65a9709955837f0a"}
Jul 26 18:03:51.835061 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:51.835014    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-679f49c568-4rwtq" podStartSLOduration=1.255606887 podStartE2EDuration="3.83499948s" totalImagesPullingTime="2.579392593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:03:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:03:51.833300842 +0000 UTC m=+386.023330596" watchObservedRunningTime="2026-07-26 18:03:51.83499948 +0000 UTC m=+386.025029237"
Jul 26 18:03:55.334503 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.334470    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"]
Jul 26 18:03:55.337596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.337577    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.340347 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.340321    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 26 18:03:55.340463 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.340358    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 26 18:03:55.340463 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.340323    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 26 18:03:55.340616 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.340597    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cxvvc\""
Jul 26 18:03:55.340722 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.340691    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 26 18:03:55.352024 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.352005    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"]
Jul 26 18:03:55.504227 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504190    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.504227 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504234    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.504450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504261    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.504450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504287    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thw9r\" (UniqueName: \"kubernetes.io/projected/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-kube-api-access-thw9r\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.504450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504310    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.504450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504346    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.504450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.504382    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.604805 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604714    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.604805 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604770    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.604805 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thw9r\" (UniqueName: \"kubernetes.io/projected/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-kube-api-access-thw9r\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.605116 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604821    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.605116 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604839    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.605116 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604855    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.605116 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.604883    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.605573 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.605539    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.607138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.607116    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.607138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.607131    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.607566 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.607535    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.607649 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.607572    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.619261 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.619237    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.619752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.619732    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thw9r\" (UniqueName: \"kubernetes.io/projected/c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c-kube-api-access-thw9r\") pod \"istiod-openshift-gateway-5677ccb49b-6pkfm\" (UID: \"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.646531 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.646503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:55.783391 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.783360    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"]
Jul 26 18:03:55.786505 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:03:55.786471    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ac5cdb_425c_49ad_8e40_a4dba9bcc07c.slice/crio-ca2029e25ac4b044db582f513c518efaeddae127525e2df5e21fd1afd72a60e8 WatchSource:0}: Error finding container ca2029e25ac4b044db582f513c518efaeddae127525e2df5e21fd1afd72a60e8: Status 404 returned error can't find the container with id ca2029e25ac4b044db582f513c518efaeddae127525e2df5e21fd1afd72a60e8
Jul 26 18:03:55.834858 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:55.834825    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm" event={"ID":"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c","Type":"ContainerStarted","Data":"ca2029e25ac4b044db582f513c518efaeddae127525e2df5e21fd1afd72a60e8"}
Jul 26 18:03:58.593276 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.593247    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"]
Jul 26 18:03:58.599810 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.599785    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:58.602915 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.602891    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9mm9c\""
Jul 26 18:03:58.603312 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.603289    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 26 18:03:58.608284 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.608255    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"]
Jul 26 18:03:58.732766 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.732735    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bl9f\" (UniqueName: \"kubernetes.io/projected/5ab06719-c410-4027-94ea-30365da8cfb7-kube-api-access-9bl9f\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:58.732933 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.732872    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:58.834022 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.833987    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bl9f\" (UniqueName: \"kubernetes.io/projected/5ab06719-c410-4027-94ea-30365da8cfb7-kube-api-access-9bl9f\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:58.834183 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.834084    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:58.834385 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:03:58.834199    2565 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 26 18:03:58.834385 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:03:58.834270    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert podName:5ab06719-c410-4027-94ea-30365da8cfb7 nodeName:}" failed. No retries permitted until 2026-07-26 18:03:59.334246917 +0000 UTC m=+393.524276652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert") pod "llmisvc-controller-manager-777c6bd7d5-cxhmm" (UID: "5ab06719-c410-4027-94ea-30365da8cfb7") : secret "llmisvc-webhook-server-cert" not found
Jul 26 18:03:58.852268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.852208    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bl9f\" (UniqueName: \"kubernetes.io/projected/5ab06719-c410-4027-94ea-30365da8cfb7-kube-api-access-9bl9f\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:58.919418 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.919374    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 26 18:03:58.919503 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:58.919453    2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Jul 26 18:03:59.337794 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:59.337744    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:03:59.338108 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:03:59.337922    2565 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 26 18:03:59.338108 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:03:59.338005    2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert podName:5ab06719-c410-4027-94ea-30365da8cfb7 nodeName:}" failed. No retries permitted until 2026-07-26 18:04:00.337984715 +0000 UTC m=+394.528014466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert") pod "llmisvc-controller-manager-777c6bd7d5-cxhmm" (UID: "5ab06719-c410-4027-94ea-30365da8cfb7") : secret "llmisvc-webhook-server-cert" not found
Jul 26 18:03:59.853491 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:59.853457    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm" event={"ID":"c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c","Type":"ContainerStarted","Data":"6e2081aff2bdb4a40b38ae8c47ec311a16977be8a60c81cef2624b9e83b28ba4"}
Jul 26 18:03:59.853949 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:59.853694    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:03:59.855088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:59.855055    2565 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-6pkfm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 26 18:03:59.855204 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:59.855116    2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm" podUID="c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 26 18:03:59.933222 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:03:59.933170    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm" podStartSLOduration=1.8024076249999998 podStartE2EDuration="4.933130488s" totalImagesPullingTime="3.130722863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:03:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:03:59.930073543 +0000 UTC m=+394.120103297" watchObservedRunningTime="2026-07-26 18:03:59.933130488 +0000 UTC m=+394.123160242"
Jul 26 18:04:00.346145 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:00.346104    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:04:00.348530 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:00.348509    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ab06719-c410-4027-94ea-30365da8cfb7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cxhmm\" (UID: \"5ab06719-c410-4027-94ea-30365da8cfb7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:04:00.414435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:00.414397    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:04:00.541566 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:00.541516    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"]
Jul 26 18:04:00.543502 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:04:00.543475    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ab06719_c410_4027_94ea_30365da8cfb7.slice/crio-3fb4b615092dd98dc4345684b472c0e044abb7a54a60a427241c097bc072aae7 WatchSource:0}: Error finding container 3fb4b615092dd98dc4345684b472c0e044abb7a54a60a427241c097bc072aae7: Status 404 returned error can't find the container with id 3fb4b615092dd98dc4345684b472c0e044abb7a54a60a427241c097bc072aae7
Jul 26 18:04:00.859365 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:00.859328    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm" event={"ID":"5ab06719-c410-4027-94ea-30365da8cfb7","Type":"ContainerStarted","Data":"3fb4b615092dd98dc4345684b472c0e044abb7a54a60a427241c097bc072aae7"}
Jul 26 18:04:00.860190 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:00.860170    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-6pkfm"
Jul 26 18:04:03.879244 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:03.879206    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm" event={"ID":"5ab06719-c410-4027-94ea-30365da8cfb7","Type":"ContainerStarted","Data":"0ca4702ecde6d14e0d3fbda23575785ee8aaba464c49a1ac9cf0cbfd9d4832a1"}
Jul 26 18:04:03.879736 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:03.879349    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:04:03.896579 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:03.896525    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm" podStartSLOduration=3.22977653 podStartE2EDuration="5.89651088s" totalImagesPullingTime="2.66673435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:04:03.895468925 +0000 UTC m=+398.085498678" watchObservedRunningTime="2026-07-26 18:04:03.89651088 +0000 UTC m=+398.086540634"
Jul 26 18:04:34.885739 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:04:34.885710    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cxhmm"
Jul 26 18:05:04.941761 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:04.941663    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"]
Jul 26 18:05:04.945180 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:04.945163    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:04.948048 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:04.948029    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 18:05:04.949094 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:04.949065    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-szp9z\""
Jul 26 18:05:04.949205 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:04.949088    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 18:05:04.953555 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:04.953534    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"]
Jul 26 18:05:05.044528 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.044498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.044645 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.044575    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w76rr\" (UniqueName: \"kubernetes.io/projected/6f73f2d9-9b81-4afa-ac6c-634a620a2568-kube-api-access-w76rr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.044645 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.044615    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.145020 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.144989    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.145193 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.145039    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w76rr\" (UniqueName: \"kubernetes.io/projected/6f73f2d9-9b81-4afa-ac6c-634a620a2568-kube-api-access-w76rr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.145193 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.145071    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.145400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.145381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.145445 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.145415    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.153886 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.153855    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w76rr\" (UniqueName: \"kubernetes.io/projected/6f73f2d9-9b81-4afa-ac6c-634a620a2568-kube-api-access-w76rr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.254605 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.254570    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:05.384047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.384018    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"]
Jul 26 18:05:05.387081 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:05.387054    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f73f2d9_9b81_4afa_ac6c_634a620a2568.slice/crio-e0952d59a1c8499c8051f12ab268bee11757b6035045d80d12e60f41f90992ee WatchSource:0}: Error finding container e0952d59a1c8499c8051f12ab268bee11757b6035045d80d12e60f41f90992ee: Status 404 returned error can't find the container with id e0952d59a1c8499c8051f12ab268bee11757b6035045d80d12e60f41f90992ee
Jul 26 18:05:05.539398 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.539317    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"]
Jul 26 18:05:05.542507 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.542491    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.550754 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.550728    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"]
Jul 26 18:05:05.652672 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.648678    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.652672 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.648754    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.652672 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.648822    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqn6d\" (UniqueName: \"kubernetes.io/projected/fe602531-96a0-46f1-8d9f-0904ff48cc37-kube-api-access-mqn6d\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.749805 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.749764    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.749805 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.749803    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.750087 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.749905    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqn6d\" (UniqueName: \"kubernetes.io/projected/fe602531-96a0-46f1-8d9f-0904ff48cc37-kube-api-access-mqn6d\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.750188 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.750167    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.750226 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.750197    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.758938 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.758915    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqn6d\" (UniqueName: \"kubernetes.io/projected/fe602531-96a0-46f1-8d9f-0904ff48cc37-kube-api-access-mqn6d\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.855734 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.855629    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:05.940205 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.940174    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"]
Jul 26 18:05:05.943654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.943634    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:05.953348 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.953309    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"]
Jul 26 18:05:05.989131 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:05.989107    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"]
Jul 26 18:05:05.991506 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:05.991477    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe602531_96a0_46f1_8d9f_0904ff48cc37.slice/crio-3d007659d4448ae84c59dc3b126d250f8d10a2f0bae9fe78aafbe9796616b28e WatchSource:0}: Error finding container 3d007659d4448ae84c59dc3b126d250f8d10a2f0bae9fe78aafbe9796616b28e: Status 404 returned error can't find the container with id 3d007659d4448ae84c59dc3b126d250f8d10a2f0bae9fe78aafbe9796616b28e
Jul 26 18:05:06.051356 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.051319    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8t5b\" (UniqueName: \"kubernetes.io/projected/a86ef9c3-62ac-4c24-bab1-1864a788a037-kube-api-access-k8t5b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.051476 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.051403    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.051476 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.051440    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.139512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.139433    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerID="a89e276a875f952c05ea3b09960ffac4f6194b4a1bdfc09bfcdb05fd4d2c21b5" exitCode=0
Jul 26 18:05:06.139648 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.139512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4" event={"ID":"6f73f2d9-9b81-4afa-ac6c-634a620a2568","Type":"ContainerDied","Data":"a89e276a875f952c05ea3b09960ffac4f6194b4a1bdfc09bfcdb05fd4d2c21b5"}
Jul 26 18:05:06.139648 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.139572    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4" event={"ID":"6f73f2d9-9b81-4afa-ac6c-634a620a2568","Type":"ContainerStarted","Data":"e0952d59a1c8499c8051f12ab268bee11757b6035045d80d12e60f41f90992ee"}
Jul 26 18:05:06.141201 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.141170    2565 generic.go:356] "Generic (PLEG): container finished" podID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerID="d05251ace8451708be08574c2d3217bf23fbc09717e4ff3385f808a1e2f37631" exitCode=0
Jul 26 18:05:06.141320 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.141204    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt" event={"ID":"fe602531-96a0-46f1-8d9f-0904ff48cc37","Type":"ContainerDied","Data":"d05251ace8451708be08574c2d3217bf23fbc09717e4ff3385f808a1e2f37631"}
Jul 26 18:05:06.141320 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.141254    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt" event={"ID":"fe602531-96a0-46f1-8d9f-0904ff48cc37","Type":"ContainerStarted","Data":"3d007659d4448ae84c59dc3b126d250f8d10a2f0bae9fe78aafbe9796616b28e"}
Jul 26 18:05:06.152280 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.152257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.152403 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.152317    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8t5b\" (UniqueName: \"kubernetes.io/projected/a86ef9c3-62ac-4c24-bab1-1864a788a037-kube-api-access-k8t5b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.152403 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.152376    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.152631 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.152612    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.152731 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.152694    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.162494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.162472    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8t5b\" (UniqueName: \"kubernetes.io/projected/a86ef9c3-62ac-4c24-bab1-1864a788a037-kube-api-access-k8t5b\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.253446 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.253407    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:06.334628 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.334598    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"]
Jul 26 18:05:06.338235 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.338213    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.347083 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.347014    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"]
Jul 26 18:05:06.388470 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.388439    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"]
Jul 26 18:05:06.391161 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:06.391099    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86ef9c3_62ac_4c24_bab1_1864a788a037.slice/crio-8de8be0a93596242925af41333658c7140f2baf153b0c766d0cc05496cc208b5 WatchSource:0}: Error finding container 8de8be0a93596242925af41333658c7140f2baf153b0c766d0cc05496cc208b5: Status 404 returned error can't find the container with id 8de8be0a93596242925af41333658c7140f2baf153b0c766d0cc05496cc208b5
Jul 26 18:05:06.455146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.455113    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.455273 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.455152    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcjdn\" (UniqueName: \"kubernetes.io/projected/c78d29ad-052c-42ef-994d-c71bb0936496-kube-api-access-gcjdn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.455273 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.455188    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.556057 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.556017    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.556236 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.556061    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcjdn\" (UniqueName: \"kubernetes.io/projected/c78d29ad-052c-42ef-994d-c71bb0936496-kube-api-access-gcjdn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.556236 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.556094    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.556470 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.556451    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.556527 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.556503    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.564460 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.564435    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcjdn\" (UniqueName: \"kubernetes.io/projected/c78d29ad-052c-42ef-994d-c71bb0936496-kube-api-access-gcjdn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.650845 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.650765    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:06.781188 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:06.781157    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"]
Jul 26 18:05:06.800959 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:06.800931    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78d29ad_052c_42ef_994d_c71bb0936496.slice/crio-e8868a121f817740502fbda890cb3b09f9939d4044593d40aa219f9232c4555b WatchSource:0}: Error finding container e8868a121f817740502fbda890cb3b09f9939d4044593d40aa219f9232c4555b: Status 404 returned error can't find the container with id e8868a121f817740502fbda890cb3b09f9939d4044593d40aa219f9232c4555b
Jul 26 18:05:07.154842 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.154810    2565 generic.go:356] "Generic (PLEG): container finished" podID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerID="69de367687239ea99b8d73b419ca4eb08544fb7dd0357b766b311ff9ff91df43" exitCode=0
Jul 26 18:05:07.155261 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.154886    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp" event={"ID":"a86ef9c3-62ac-4c24-bab1-1864a788a037","Type":"ContainerDied","Data":"69de367687239ea99b8d73b419ca4eb08544fb7dd0357b766b311ff9ff91df43"}
Jul 26 18:05:07.155261 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.154973    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp" event={"ID":"a86ef9c3-62ac-4c24-bab1-1864a788a037","Type":"ContainerStarted","Data":"8de8be0a93596242925af41333658c7140f2baf153b0c766d0cc05496cc208b5"}
Jul 26 18:05:07.156981 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.156957    2565 generic.go:356] "Generic (PLEG): container finished" podID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerID="2e455ecead7ab3f0859b5b298ae17d7a520f64a13a6733e5cf0b23007672aaec" exitCode=0
Jul 26 18:05:07.157130 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.157034    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt" event={"ID":"fe602531-96a0-46f1-8d9f-0904ff48cc37","Type":"ContainerDied","Data":"2e455ecead7ab3f0859b5b298ae17d7a520f64a13a6733e5cf0b23007672aaec"}
Jul 26 18:05:07.158663 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.158645    2565 generic.go:356] "Generic (PLEG): container finished" podID="c78d29ad-052c-42ef-994d-c71bb0936496" containerID="f74b2557cdbeea7b78fafa7c9cf2eb76b76163fa7bd8c1991e36c3878856dcd3" exitCode=0
Jul 26 18:05:07.158757 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.158717    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j" event={"ID":"c78d29ad-052c-42ef-994d-c71bb0936496","Type":"ContainerDied","Data":"f74b2557cdbeea7b78fafa7c9cf2eb76b76163fa7bd8c1991e36c3878856dcd3"}
Jul 26 18:05:07.158757 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:07.158749    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j" event={"ID":"c78d29ad-052c-42ef-994d-c71bb0936496","Type":"ContainerStarted","Data":"e8868a121f817740502fbda890cb3b09f9939d4044593d40aa219f9232c4555b"}
Jul 26 18:05:08.164735 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.164686    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerID="b7fbc766ed43b24a1cac8e3f3762498d2d18a6f167efd3e2a45db6e013abea54" exitCode=0
Jul 26 18:05:08.165166 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.164734    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4" event={"ID":"6f73f2d9-9b81-4afa-ac6c-634a620a2568","Type":"ContainerDied","Data":"b7fbc766ed43b24a1cac8e3f3762498d2d18a6f167efd3e2a45db6e013abea54"}
Jul 26 18:05:08.166809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.166751    2565 generic.go:356] "Generic (PLEG): container finished" podID="c78d29ad-052c-42ef-994d-c71bb0936496" containerID="ab6e182e515a21773feab87dcae8a94e41599ec0bd154412189dbb3ef10b47fc" exitCode=0
Jul 26 18:05:08.166906 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.166850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j" event={"ID":"c78d29ad-052c-42ef-994d-c71bb0936496","Type":"ContainerDied","Data":"ab6e182e515a21773feab87dcae8a94e41599ec0bd154412189dbb3ef10b47fc"}
Jul 26 18:05:08.168838 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.168816    2565 generic.go:356] "Generic (PLEG): container finished" podID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerID="758248e3aa67677f6d6e89788c7314fc757e3f8fe25365e3f70283b57ec8eda8" exitCode=0
Jul 26 18:05:08.168918 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.168898    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp" event={"ID":"a86ef9c3-62ac-4c24-bab1-1864a788a037","Type":"ContainerDied","Data":"758248e3aa67677f6d6e89788c7314fc757e3f8fe25365e3f70283b57ec8eda8"}
Jul 26 18:05:08.171204 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.171178    2565 generic.go:356] "Generic (PLEG): container finished" podID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerID="a38d87ca1d5c8a4de793cb2f54fb62a68914e3024b5d43fc9c3ac091e60bc1dd" exitCode=0
Jul 26 18:05:08.171289 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:08.171209    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt" event={"ID":"fe602531-96a0-46f1-8d9f-0904ff48cc37","Type":"ContainerDied","Data":"a38d87ca1d5c8a4de793cb2f54fb62a68914e3024b5d43fc9c3ac091e60bc1dd"}
Jul 26 18:05:09.178456 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.178426    2565 generic.go:356] "Generic (PLEG): container finished" podID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerID="4cc6ccc4f38c27c758a692e91940a5c6000f794a6994eefc13be7caeca7783c3" exitCode=0
Jul 26 18:05:09.178831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.178507    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4" event={"ID":"6f73f2d9-9b81-4afa-ac6c-634a620a2568","Type":"ContainerDied","Data":"4cc6ccc4f38c27c758a692e91940a5c6000f794a6994eefc13be7caeca7783c3"}
Jul 26 18:05:09.180475 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.180456    2565 generic.go:356] "Generic (PLEG): container finished" podID="c78d29ad-052c-42ef-994d-c71bb0936496" containerID="f16c28a16e04bbaebf6f2f7bf4c5179362c318ac971df2ef55f70f9fb4d46489" exitCode=0
Jul 26 18:05:09.180589 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.180513    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j" event={"ID":"c78d29ad-052c-42ef-994d-c71bb0936496","Type":"ContainerDied","Data":"f16c28a16e04bbaebf6f2f7bf4c5179362c318ac971df2ef55f70f9fb4d46489"}
Jul 26 18:05:09.182434 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.182414    2565 generic.go:356] "Generic (PLEG): container finished" podID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerID="703ee6e4cb0a86df3b9537a4fa0c666653d3ff0e60c9185a604314921727e22f" exitCode=0
Jul 26 18:05:09.182538 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.182479    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp" event={"ID":"a86ef9c3-62ac-4c24-bab1-1864a788a037","Type":"ContainerDied","Data":"703ee6e4cb0a86df3b9537a4fa0c666653d3ff0e60c9185a604314921727e22f"}
Jul 26 18:05:09.310329 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.310306    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:09.481374 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.481347    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-bundle\") pod \"fe602531-96a0-46f1-8d9f-0904ff48cc37\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") "
Jul 26 18:05:09.481526 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.481386    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqn6d\" (UniqueName: \"kubernetes.io/projected/fe602531-96a0-46f1-8d9f-0904ff48cc37-kube-api-access-mqn6d\") pod \"fe602531-96a0-46f1-8d9f-0904ff48cc37\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") "
Jul 26 18:05:09.481526 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.481410    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-util\") pod \"fe602531-96a0-46f1-8d9f-0904ff48cc37\" (UID: \"fe602531-96a0-46f1-8d9f-0904ff48cc37\") "
Jul 26 18:05:09.481902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.481873    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-bundle" (OuterVolumeSpecName: "bundle") pod "fe602531-96a0-46f1-8d9f-0904ff48cc37" (UID: "fe602531-96a0-46f1-8d9f-0904ff48cc37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:09.483583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.483555    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe602531-96a0-46f1-8d9f-0904ff48cc37-kube-api-access-mqn6d" (OuterVolumeSpecName: "kube-api-access-mqn6d") pod "fe602531-96a0-46f1-8d9f-0904ff48cc37" (UID: "fe602531-96a0-46f1-8d9f-0904ff48cc37"). InnerVolumeSpecName "kube-api-access-mqn6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:05:09.486907 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.486888    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-util" (OuterVolumeSpecName: "util") pod "fe602531-96a0-46f1-8d9f-0904ff48cc37" (UID: "fe602531-96a0-46f1-8d9f-0904ff48cc37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:09.582779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.582746    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:09.582779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.582775    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqn6d\" (UniqueName: \"kubernetes.io/projected/fe602531-96a0-46f1-8d9f-0904ff48cc37-kube-api-access-mqn6d\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:09.582779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:09.582788    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe602531-96a0-46f1-8d9f-0904ff48cc37-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.194268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.194234    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt"
Jul 26 18:05:10.194722 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.194270    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt" event={"ID":"fe602531-96a0-46f1-8d9f-0904ff48cc37","Type":"ContainerDied","Data":"3d007659d4448ae84c59dc3b126d250f8d10a2f0bae9fe78aafbe9796616b28e"}
Jul 26 18:05:10.194722 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.194352    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d007659d4448ae84c59dc3b126d250f8d10a2f0bae9fe78aafbe9796616b28e"
Jul 26 18:05:10.332631 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.332603    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:10.374735 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.374690    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:10.377285 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.377264    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:10.491968 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.491931    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8t5b\" (UniqueName: \"kubernetes.io/projected/a86ef9c3-62ac-4c24-bab1-1864a788a037-kube-api-access-k8t5b\") pod \"a86ef9c3-62ac-4c24-bab1-1864a788a037\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") "
Jul 26 18:05:10.492147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.491976    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-util\") pod \"c78d29ad-052c-42ef-994d-c71bb0936496\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") "
Jul 26 18:05:10.492147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.491995    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-bundle\") pod \"c78d29ad-052c-42ef-994d-c71bb0936496\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") "
Jul 26 18:05:10.492147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492067    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-util\") pod \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") "
Jul 26 18:05:10.492147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492098    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-util\") pod \"a86ef9c3-62ac-4c24-bab1-1864a788a037\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") "
Jul 26 18:05:10.492147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492120    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-bundle\") pod \"a86ef9c3-62ac-4c24-bab1-1864a788a037\" (UID: \"a86ef9c3-62ac-4c24-bab1-1864a788a037\") "
Jul 26 18:05:10.492421 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492152    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcjdn\" (UniqueName: \"kubernetes.io/projected/c78d29ad-052c-42ef-994d-c71bb0936496-kube-api-access-gcjdn\") pod \"c78d29ad-052c-42ef-994d-c71bb0936496\" (UID: \"c78d29ad-052c-42ef-994d-c71bb0936496\") "
Jul 26 18:05:10.492421 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492180    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-bundle\") pod \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") "
Jul 26 18:05:10.492421 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492225    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w76rr\" (UniqueName: \"kubernetes.io/projected/6f73f2d9-9b81-4afa-ac6c-634a620a2568-kube-api-access-w76rr\") pod \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\" (UID: \"6f73f2d9-9b81-4afa-ac6c-634a620a2568\") "
Jul 26 18:05:10.493289 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492628    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-bundle" (OuterVolumeSpecName: "bundle") pod "a86ef9c3-62ac-4c24-bab1-1864a788a037" (UID: "a86ef9c3-62ac-4c24-bab1-1864a788a037"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:10.493289 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.492862    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-bundle" (OuterVolumeSpecName: "bundle") pod "c78d29ad-052c-42ef-994d-c71bb0936496" (UID: "c78d29ad-052c-42ef-994d-c71bb0936496"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:10.493289 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.493239    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-bundle" (OuterVolumeSpecName: "bundle") pod "6f73f2d9-9b81-4afa-ac6c-634a620a2568" (UID: "6f73f2d9-9b81-4afa-ac6c-634a620a2568"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:10.494674 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.494647    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a86ef9c3-62ac-4c24-bab1-1864a788a037-kube-api-access-k8t5b" (OuterVolumeSpecName: "kube-api-access-k8t5b") pod "a86ef9c3-62ac-4c24-bab1-1864a788a037" (UID: "a86ef9c3-62ac-4c24-bab1-1864a788a037"). InnerVolumeSpecName "kube-api-access-k8t5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:05:10.494936 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.494738    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c78d29ad-052c-42ef-994d-c71bb0936496-kube-api-access-gcjdn" (OuterVolumeSpecName: "kube-api-access-gcjdn") pod "c78d29ad-052c-42ef-994d-c71bb0936496" (UID: "c78d29ad-052c-42ef-994d-c71bb0936496"). InnerVolumeSpecName "kube-api-access-gcjdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:05:10.494936 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.494779    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f73f2d9-9b81-4afa-ac6c-634a620a2568-kube-api-access-w76rr" (OuterVolumeSpecName: "kube-api-access-w76rr") pod "6f73f2d9-9b81-4afa-ac6c-634a620a2568" (UID: "6f73f2d9-9b81-4afa-ac6c-634a620a2568"). InnerVolumeSpecName "kube-api-access-w76rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:05:10.498538 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.498514    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-util" (OuterVolumeSpecName: "util") pod "6f73f2d9-9b81-4afa-ac6c-634a620a2568" (UID: "6f73f2d9-9b81-4afa-ac6c-634a620a2568"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:10.498750 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.498730    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-util" (OuterVolumeSpecName: "util") pod "c78d29ad-052c-42ef-994d-c71bb0936496" (UID: "c78d29ad-052c-42ef-994d-c71bb0936496"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:10.498804 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.498757    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-util" (OuterVolumeSpecName: "util") pod "a86ef9c3-62ac-4c24-bab1-1864a788a037" (UID: "a86ef9c3-62ac-4c24-bab1-1864a788a037"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:10.592835 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592798    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.592835 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592826    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.592835 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592835    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a86ef9c3-62ac-4c24-bab1-1864a788a037-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.592835 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592844    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcjdn\" (UniqueName: \"kubernetes.io/projected/c78d29ad-052c-42ef-994d-c71bb0936496-kube-api-access-gcjdn\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.593282 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592854    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f73f2d9-9b81-4afa-ac6c-634a620a2568-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.593282 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592863    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w76rr\" (UniqueName: \"kubernetes.io/projected/6f73f2d9-9b81-4afa-ac6c-634a620a2568-kube-api-access-w76rr\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.593282 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592872    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8t5b\" (UniqueName: \"kubernetes.io/projected/a86ef9c3-62ac-4c24-bab1-1864a788a037-kube-api-access-k8t5b\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.593282 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592881    2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-util\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:10.593282 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:10.592889    2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c78d29ad-052c-42ef-994d-c71bb0936496-bundle\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:11.202023 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.201992    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp"
Jul 26 18:05:11.202023 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.202021    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp" event={"ID":"a86ef9c3-62ac-4c24-bab1-1864a788a037","Type":"ContainerDied","Data":"8de8be0a93596242925af41333658c7140f2baf153b0c766d0cc05496cc208b5"}
Jul 26 18:05:11.202470 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.202069    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8de8be0a93596242925af41333658c7140f2baf153b0c766d0cc05496cc208b5"
Jul 26 18:05:11.203802 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.203782    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4" event={"ID":"6f73f2d9-9b81-4afa-ac6c-634a620a2568","Type":"ContainerDied","Data":"e0952d59a1c8499c8051f12ab268bee11757b6035045d80d12e60f41f90992ee"}
Jul 26 18:05:11.203802 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.203796    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4"
Jul 26 18:05:11.203961 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.203820    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e0952d59a1c8499c8051f12ab268bee11757b6035045d80d12e60f41f90992ee"
Jul 26 18:05:11.205538 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.205519    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j" event={"ID":"c78d29ad-052c-42ef-994d-c71bb0936496","Type":"ContainerDied","Data":"e8868a121f817740502fbda890cb3b09f9939d4044593d40aa219f9232c4555b"}
Jul 26 18:05:11.205626 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.205553    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8868a121f817740502fbda890cb3b09f9939d4044593d40aa219f9232c4555b"
Jul 26 18:05:11.205626 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:11.205526    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j"
Jul 26 18:05:20.792811 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.792782    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"]
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793027    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793037    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793048    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793054    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793060    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793065    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793073    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793078    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793086    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793090    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793098    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793103    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793108    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793113    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793120    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793125    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793132    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793137    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="pull"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793144    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793149    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793155    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793160    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="util"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793167    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="extract"
Jul 26 18:05:20.793163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793172    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="extract"
Jul 26 18:05:20.793809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793211    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78d29ad-052c-42ef-994d-c71bb0936496" containerName="extract"
Jul 26 18:05:20.793809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793218    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f73f2d9-9b81-4afa-ac6c-634a620a2568" containerName="extract"
Jul 26 18:05:20.793809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793223    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a86ef9c3-62ac-4c24-bab1-1864a788a037" containerName="extract"
Jul 26 18:05:20.793809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.793230    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe602531-96a0-46f1-8d9f-0904ff48cc37" containerName="extract"
Jul 26 18:05:20.795957 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.795940    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:20.798917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.798900    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t2d9w\""
Jul 26 18:05:20.799166 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.799151    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 18:05:20.799508 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.799494    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 18:05:20.822141 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.822119    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"]
Jul 26 18:05:20.975600 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.975563    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/289ebfc0-ef1c-422b-882a-00f94da4bbe8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:20.975797 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:20.975613    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6jwl\" (UniqueName: \"kubernetes.io/projected/289ebfc0-ef1c-422b-882a-00f94da4bbe8-kube-api-access-n6jwl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:21.076935 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:21.076843    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6jwl\" (UniqueName: \"kubernetes.io/projected/289ebfc0-ef1c-422b-882a-00f94da4bbe8-kube-api-access-n6jwl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:21.076935 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:21.076933    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/289ebfc0-ef1c-422b-882a-00f94da4bbe8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:21.077258 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:21.077244    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/289ebfc0-ef1c-422b-882a-00f94da4bbe8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:21.096401 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:21.096364    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6jwl\" (UniqueName: \"kubernetes.io/projected/289ebfc0-ef1c-422b-882a-00f94da4bbe8-kube-api-access-n6jwl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:21.105465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:21.105445    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:21.254856 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:21.254836    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"]
Jul 26 18:05:21.257112 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:21.257085    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289ebfc0_ef1c_422b_882a_00f94da4bbe8.slice/crio-7136c25f6a38b0b13a66df8413daa61bba1fa65dc5ca46c963c265722cf7c432 WatchSource:0}: Error finding container 7136c25f6a38b0b13a66df8413daa61bba1fa65dc5ca46c963c265722cf7c432: Status 404 returned error can't find the container with id 7136c25f6a38b0b13a66df8413daa61bba1fa65dc5ca46c963c265722cf7c432
Jul 26 18:05:22.257907 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:22.257870    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv" event={"ID":"289ebfc0-ef1c-422b-882a-00f94da4bbe8","Type":"ContainerStarted","Data":"7136c25f6a38b0b13a66df8413daa61bba1fa65dc5ca46c963c265722cf7c432"}
Jul 26 18:05:25.543713 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.543667    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"]
Jul 26 18:05:25.547155 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.547132    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:25.551436 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.551410    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-k6zm9\""
Jul 26 18:05:25.567432 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.567403    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"]
Jul 26 18:05:25.612870 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.612834    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl92g\" (UniqueName: \"kubernetes.io/projected/864fac41-3d29-402f-ad83-0e16c41b4f2a-kube-api-access-hl92g\") pod \"authorino-operator-75cc8d5cb7-wcqhn\" (UID: \"864fac41-3d29-402f-ad83-0e16c41b4f2a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:25.713466 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.713424    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl92g\" (UniqueName: \"kubernetes.io/projected/864fac41-3d29-402f-ad83-0e16c41b4f2a-kube-api-access-hl92g\") pod \"authorino-operator-75cc8d5cb7-wcqhn\" (UID: \"864fac41-3d29-402f-ad83-0e16c41b4f2a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:25.732484 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.732444    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl92g\" (UniqueName: \"kubernetes.io/projected/864fac41-3d29-402f-ad83-0e16c41b4f2a-kube-api-access-hl92g\") pod \"authorino-operator-75cc8d5cb7-wcqhn\" (UID: \"864fac41-3d29-402f-ad83-0e16c41b4f2a\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:25.859369 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:25.859284    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:27.070217 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:27.070198    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"]
Jul 26 18:05:27.073554 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:27.073531    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864fac41_3d29_402f_ad83_0e16c41b4f2a.slice/crio-11608404ef5d93fa65d757bd40ac63d38af248bfc39b60eea4dd1bcf592916ed WatchSource:0}: Error finding container 11608404ef5d93fa65d757bd40ac63d38af248bfc39b60eea4dd1bcf592916ed: Status 404 returned error can't find the container with id 11608404ef5d93fa65d757bd40ac63d38af248bfc39b60eea4dd1bcf592916ed
Jul 26 18:05:27.285173 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:27.285136    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv" event={"ID":"289ebfc0-ef1c-422b-882a-00f94da4bbe8","Type":"ContainerStarted","Data":"cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482"}
Jul 26 18:05:27.285381 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:27.285301    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:27.286360 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:27.286336    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn" event={"ID":"864fac41-3d29-402f-ad83-0e16c41b4f2a","Type":"ContainerStarted","Data":"11608404ef5d93fa65d757bd40ac63d38af248bfc39b60eea4dd1bcf592916ed"}
Jul 26 18:05:27.335761 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:27.335670    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv" podStartSLOduration=1.6054773340000001 podStartE2EDuration="7.335650874s" totalImagesPullingTime="5.73017354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:05:27.33350014 +0000 UTC m=+481.523529895" watchObservedRunningTime="2026-07-26 18:05:27.335650874 +0000 UTC m=+481.525680629"
Jul 26 18:05:30.303160 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:30.303120    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn" event={"ID":"864fac41-3d29-402f-ad83-0e16c41b4f2a","Type":"ContainerStarted","Data":"a486780b2886648e2a2f719e2bea3ef6dc56660f1a9903356b7ffcd83197d952"}
Jul 26 18:05:30.303622 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:30.303195    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:30.342575 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:30.342524    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn" podStartSLOduration=3.072200417 podStartE2EDuration="5.342507807s" totalImagesPullingTime="2.27030739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:05:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:05:30.341035323 +0000 UTC m=+484.531065079" watchObservedRunningTime="2026-07-26 18:05:30.342507807 +0000 UTC m=+484.532537562"
Jul 26 18:05:38.293850 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:38.293816    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:40.691555 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.691522    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"]
Jul 26 18:05:40.691988 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.691769    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" containerName="manager" containerID="cri-o://cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482" gracePeriod=2
Jul 26 18:05:40.709688 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.709663    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"]
Jul 26 18:05:40.721779 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:40.721749    2565 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"289ebfc0-ef1c-422b-882a-00f94da4bbe8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\": pods \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:40.791947 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.791918    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"]
Jul 26 18:05:40.792193 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.792182    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" containerName="manager"
Jul 26 18:05:40.792235 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.792195    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" containerName="manager"
Jul 26 18:05:40.792268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.792252    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" containerName="manager"
Jul 26 18:05:40.795147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.795131    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:05:40.798389 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.798360    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hj5wg\""
Jul 26 18:05:40.812873 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.812851    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"]
Jul 26 18:05:40.829337 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.829309    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9t5b\" (UniqueName: \"kubernetes.io/projected/c10be1d9-754e-4151-8abc-c362270944c3-kube-api-access-w9t5b\") pod \"limitador-operator-controller-manager-859d7b78bd-qdnfh\" (UID: \"c10be1d9-754e-4151-8abc-c362270944c3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:05:40.833952 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:40.833924    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:40.930689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.930663    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9t5b\" (UniqueName: \"kubernetes.io/projected/c10be1d9-754e-4151-8abc-c362270944c3-kube-api-access-w9t5b\") pod \"limitador-operator-controller-manager-859d7b78bd-qdnfh\" (UID: \"c10be1d9-754e-4151-8abc-c362270944c3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:05:40.941352 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.941322    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9t5b\" (UniqueName: \"kubernetes.io/projected/c10be1d9-754e-4151-8abc-c362270944c3-kube-api-access-w9t5b\") pod \"limitador-operator-controller-manager-859d7b78bd-qdnfh\" (UID: \"c10be1d9-754e-4151-8abc-c362270944c3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:05:40.942620 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:40.942575    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:40.946132 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:40.946105    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:41.031567 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.031537    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6jwl\" (UniqueName: \"kubernetes.io/projected/289ebfc0-ef1c-422b-882a-00f94da4bbe8-kube-api-access-n6jwl\") pod \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") "
Jul 26 18:05:41.031778 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.031596    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/289ebfc0-ef1c-422b-882a-00f94da4bbe8-extensions-socket-volume\") pod \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\" (UID: \"289ebfc0-ef1c-422b-882a-00f94da4bbe8\") "
Jul 26 18:05:41.032103 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.032073    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/289ebfc0-ef1c-422b-882a-00f94da4bbe8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "289ebfc0-ef1c-422b-882a-00f94da4bbe8" (UID: "289ebfc0-ef1c-422b-882a-00f94da4bbe8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:05:41.033586 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.033559    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/289ebfc0-ef1c-422b-882a-00f94da4bbe8-kube-api-access-n6jwl" (OuterVolumeSpecName: "kube-api-access-n6jwl") pod "289ebfc0-ef1c-422b-882a-00f94da4bbe8" (UID: "289ebfc0-ef1c-422b-882a-00f94da4bbe8"). InnerVolumeSpecName "kube-api-access-n6jwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:05:41.106248 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.106211    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:05:41.133069 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.133042    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6jwl\" (UniqueName: \"kubernetes.io/projected/289ebfc0-ef1c-422b-882a-00f94da4bbe8-kube-api-access-n6jwl\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:41.133069 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.133066    2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/289ebfc0-ef1c-422b-882a-00f94da4bbe8-extensions-socket-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:05:41.246724 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.246680    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"]
Jul 26 18:05:41.248165 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:05:41.248136    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10be1d9_754e_4151_8abc_c362270944c3.slice/crio-8fbefbbfd417c9e88fbd787c74b40d4a461852b194f6adb73118df3c1dd98e5c WatchSource:0}: Error finding container 8fbefbbfd417c9e88fbd787c74b40d4a461852b194f6adb73118df3c1dd98e5c: Status 404 returned error can't find the container with id 8fbefbbfd417c9e88fbd787c74b40d4a461852b194f6adb73118df3c1dd98e5c
Jul 26 18:05:41.310270 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.310235    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wcqhn"
Jul 26 18:05:41.312738 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:41.312687    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:41.351263 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.351226    2565 generic.go:356] "Generic (PLEG): container finished" podID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" containerID="cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482" exitCode=0
Jul 26 18:05:41.351429 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.351274    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:41.351429 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.351325    2565 scope.go:117] "RemoveContainer" containerID="cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482"
Jul 26 18:05:41.352584 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.352559    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh" event={"ID":"c10be1d9-754e-4151-8abc-c362270944c3","Type":"ContainerStarted","Data":"8fbefbbfd417c9e88fbd787c74b40d4a461852b194f6adb73118df3c1dd98e5c"}
Jul 26 18:05:41.356106 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:41.356077    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:41.360838 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.360823    2565 scope.go:117] "RemoveContainer" containerID="cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482"
Jul 26 18:05:41.361095 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:41.361078    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482\": container with ID starting with cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482 not found: ID does not exist" containerID="cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482"
Jul 26 18:05:41.361145 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:41.361103    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482"} err="failed to get container status \"cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482\": rpc error: code = NotFound desc = could not find container \"cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482\": container with ID starting with cd721b7e55c81b5332b50e7b4a9bac9f3904c31ccc09e069618235e53a54d482 not found: ID does not exist"
Jul 26 18:05:41.363233 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:05:41.363214    2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-2xsfv\" is forbidden: User \"system:node:ip-10-0-133-34.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-34.ec2.internal' and this object" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2xsfv"
Jul 26 18:05:42.320074 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:42.320037    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="289ebfc0-ef1c-422b-882a-00f94da4bbe8" path="/var/lib/kubelet/pods/289ebfc0-ef1c-422b-882a-00f94da4bbe8/volumes"
Jul 26 18:05:43.365179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:43.365143    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh" event={"ID":"c10be1d9-754e-4151-8abc-c362270944c3","Type":"ContainerStarted","Data":"7973bbf7fddec62dfc6db196b8cbd668887ea623df7b893022957b7850450ec1"}
Jul 26 18:05:43.365661 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:43.365295    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:05:43.400412 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:43.400359    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh" podStartSLOduration=2.242733765 podStartE2EDuration="3.400343513s" totalImagesPullingTime="1.157609748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:05:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:05:43.399939705 +0000 UTC m=+497.589969479" watchObservedRunningTime="2026-07-26 18:05:43.400343513 +0000 UTC m=+497.590373339"
Jul 26 18:05:54.373109 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:05:54.373080    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qdnfh"
Jul 26 18:06:17.651584 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.651549    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:06:17.661975 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.661949    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.665738 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.665688    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-szp9z\""
Jul 26 18:06:17.665975 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.665957    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 26 18:06:17.678812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.678785    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:06:17.704683 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.704650    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:06:17.829108 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.829077    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/53db2922-3aa6-42af-abfc-f7400012ce5e-config-file\") pod \"limitador-limitador-54d6d988d5-bklf4\" (UID: \"53db2922-3aa6-42af-abfc-f7400012ce5e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.829268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.829116    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6qk6\" (UniqueName: \"kubernetes.io/projected/53db2922-3aa6-42af-abfc-f7400012ce5e-kube-api-access-r6qk6\") pod \"limitador-limitador-54d6d988d5-bklf4\" (UID: \"53db2922-3aa6-42af-abfc-f7400012ce5e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.929779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.929657    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6qk6\" (UniqueName: \"kubernetes.io/projected/53db2922-3aa6-42af-abfc-f7400012ce5e-kube-api-access-r6qk6\") pod \"limitador-limitador-54d6d988d5-bklf4\" (UID: \"53db2922-3aa6-42af-abfc-f7400012ce5e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.929779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.929771    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/53db2922-3aa6-42af-abfc-f7400012ce5e-config-file\") pod \"limitador-limitador-54d6d988d5-bklf4\" (UID: \"53db2922-3aa6-42af-abfc-f7400012ce5e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.930347 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.930329    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/53db2922-3aa6-42af-abfc-f7400012ce5e-config-file\") pod \"limitador-limitador-54d6d988d5-bklf4\" (UID: \"53db2922-3aa6-42af-abfc-f7400012ce5e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.945154 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.945124    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6qk6\" (UniqueName: \"kubernetes.io/projected/53db2922-3aa6-42af-abfc-f7400012ce5e-kube-api-access-r6qk6\") pod \"limitador-limitador-54d6d988d5-bklf4\" (UID: \"53db2922-3aa6-42af-abfc-f7400012ce5e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:17.972043 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:17.972001    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:18.126795 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:18.126761    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:06:18.129791 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:06:18.129766    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53db2922_3aa6_42af_abfc_f7400012ce5e.slice/crio-7ebe9825b71587ad020bbe15cca08d55911ced41a6058c7d2c2ddc24c0a14bd7 WatchSource:0}: Error finding container 7ebe9825b71587ad020bbe15cca08d55911ced41a6058c7d2c2ddc24c0a14bd7: Status 404 returned error can't find the container with id 7ebe9825b71587ad020bbe15cca08d55911ced41a6058c7d2c2ddc24c0a14bd7
Jul 26 18:06:18.526885 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:18.526847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4" event={"ID":"53db2922-3aa6-42af-abfc-f7400012ce5e","Type":"ContainerStarted","Data":"7ebe9825b71587ad020bbe15cca08d55911ced41a6058c7d2c2ddc24c0a14bd7"}
Jul 26 18:06:21.548249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:21.548205    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4" event={"ID":"53db2922-3aa6-42af-abfc-f7400012ce5e","Type":"ContainerStarted","Data":"f97cac3c736c1d5d41c2abddead891c9548de0950e35327146e889f46c983cfb"}
Jul 26 18:06:21.548630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:21.548296    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:06:21.572953 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:21.572905    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4" podStartSLOduration=2.077971763 podStartE2EDuration="4.572890028s" totalImagesPullingTime="2.494918265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:06:21.571964363 +0000 UTC m=+535.761994117" watchObservedRunningTime="2026-07-26 18:06:21.572890028 +0000 UTC m=+535.762919782"
Jul 26 18:06:32.553187 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:06:32.553148    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-bklf4"
Jul 26 18:07:34.011774 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:07:34.011736    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:07:48.024526 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:07:48.024436    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:08:00.532428 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:00.532385    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:08:16.216054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:16.216014    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:08:20.818974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:20.818942    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:08:25.604994 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.604959    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"]
Jul 26 18:08:25.608299 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.608278    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.614345 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.614321    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sc8sl\""
Jul 26 18:08:25.614486 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.614325    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 26 18:08:25.614486 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.614329    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 18:08:25.625547 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.625521    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 18:08:25.642028 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.641998    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"]
Jul 26 18:08:25.711761 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.711725    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.711924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.711775    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6hhc\" (UniqueName: \"kubernetes.io/projected/520ed8b5-bd68-4675-be8a-34f419ce2968-kube-api-access-z6hhc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.711924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.711793    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.711924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.711854    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ed8b5-bd68-4675-be8a-34f419ce2968-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.711924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.711881    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.711924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.711901    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.812831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.812798    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6hhc\" (UniqueName: \"kubernetes.io/projected/520ed8b5-bd68-4675-be8a-34f419ce2968-kube-api-access-z6hhc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.812831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.812832    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813074 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.812862    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ed8b5-bd68-4675-be8a-34f419ce2968-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813074 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.812879    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813074 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.812895    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813074 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.812950    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.813227    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813344 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.813319    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.813405 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.813370    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.815125 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.815108    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ed8b5-bd68-4675-be8a-34f419ce2968-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.815365 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.815346    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ed8b5-bd68-4675-be8a-34f419ce2968-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.821751 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.821729    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6hhc\" (UniqueName: \"kubernetes.io/projected/520ed8b5-bd68-4675-be8a-34f419ce2968-kube-api-access-z6hhc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7jsnd\" (UID: \"520ed8b5-bd68-4675-be8a-34f419ce2968\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:25.918232 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:25.918143    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:26.056308 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:26.056279    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"]
Jul 26 18:08:26.058255 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:08:26.058219    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod520ed8b5_bd68_4675_be8a_34f419ce2968.slice/crio-40c67dfef70b2479cc582f53d4e8468d0e7c77b948e443e2774f11ba7f195a91 WatchSource:0}: Error finding container 40c67dfef70b2479cc582f53d4e8468d0e7c77b948e443e2774f11ba7f195a91: Status 404 returned error can't find the container with id 40c67dfef70b2479cc582f53d4e8468d0e7c77b948e443e2774f11ba7f195a91
Jul 26 18:08:26.059658 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:26.059644    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:08:26.136144 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:26.136104    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd" event={"ID":"520ed8b5-bd68-4675-be8a-34f419ce2968","Type":"ContainerStarted","Data":"40c67dfef70b2479cc582f53d4e8468d0e7c77b948e443e2774f11ba7f195a91"}
Jul 26 18:08:27.142306 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:27.142274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd" event={"ID":"520ed8b5-bd68-4675-be8a-34f419ce2968","Type":"ContainerStarted","Data":"99369d4f876e10e92199d1126eae310edc6ea103785fdd10a83870d7b656aa1f"}
Jul 26 18:08:27.165281 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:27.165230    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd" podStartSLOduration=1.197572978 podStartE2EDuration="2.165216542s" totalImagesPullingTime="967.643564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:08:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:08:27.163869244 +0000 UTC m=+661.353898999" watchObservedRunningTime="2026-07-26 18:08:27.165216542 +0000 UTC m=+661.355246296"
Jul 26 18:08:28.229660 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:28.229630    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:08:35.918428 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:35.918393    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:35.918849 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:35.918459    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:35.930994 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:35.930971    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:36.199029 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:36.198958    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7jsnd"
Jul 26 18:08:42.325248 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:08:42.325216    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:10:50.922559 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:10:50.922481    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:11:00.523537 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:11:00.523503    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:11:11.151973 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:11:11.151939    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:12:52.744214 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:12:52.744178    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:13:00.711772 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:13:00.711733    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:13:33.011623 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:13:33.011589    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:13:41.919170 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:13:41.919134    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:13:50.715643 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:13:50.715554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:14:00.114105 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:14:00.114069    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:14:10.730175 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:14:10.730140    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:14:20.319465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:14:20.319433    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:14:31.046654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:14:31.046621    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:14:41.317601 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:14:41.317569    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:15:06.311431 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:15:06.311392    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:15:17.711274 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:15:17.711241    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:15:41.622231 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:15:41.622190    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:15:52.343882 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:15:52.343845    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:16:06.816668 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:16:06.816633    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:16:17.930076 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:16:17.930041    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:16:27.525100 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:16:27.525066    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:16:37.213980 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:16:37.213947    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:16:46.911249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:16:46.911218    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:18:35.116381 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:18:35.116290    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:18:46.616556 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:18:46.616520    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:18:55.222099 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:18:55.222061    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:19:07.644471 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:19:07.644435    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:19:18.917774 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:19:18.917739    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:19:29.360933 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:19:29.360899    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:19:38.932548 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:19:38.932513    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:19:48.318985 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:19:48.318913    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:20:00.847333 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:00.847289    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:20:09.248655 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:09.248623    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:20:20.328509 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:20.328469    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:20:44.448574 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:44.448532    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:20:51.824219 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:51.824183    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:20:56.750246 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.750210    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"]
Jul 26 18:20:56.753458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.753440    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.756901 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.756878    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-fea2be60-kserve-self-signed-certs\""
Jul 26 18:20:56.783458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.783428    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"]
Jul 26 18:20:56.840913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.840879    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-tmp-dir\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.840913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.840914    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-dshm\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.841133 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.840934    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-home\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.841133 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.841034    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b441786-9bec-4201-9aae-83a19e97ccdd-tls-certs\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.841133 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.841068    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-model-cache\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.841133 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.841086    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbzzx\" (UniqueName: \"kubernetes.io/projected/5b441786-9bec-4201-9aae-83a19e97ccdd-kube-api-access-dbzzx\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.941552 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b441786-9bec-4201-9aae-83a19e97ccdd-tls-certs\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.941552 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941556    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-model-cache\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.941840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941582    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbzzx\" (UniqueName: \"kubernetes.io/projected/5b441786-9bec-4201-9aae-83a19e97ccdd-kube-api-access-dbzzx\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.941840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941631    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-tmp-dir\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.941840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941655    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-dshm\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.941840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941678    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-home\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.942042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.941958    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-model-cache\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.942042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.942017    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-tmp-dir\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.942147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.942121    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-home\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.944051 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.944027    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-dshm\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.944191 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.944177    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b441786-9bec-4201-9aae-83a19e97ccdd-tls-certs\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:56.951202 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:56.951178    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbzzx\" (UniqueName: \"kubernetes.io/projected/5b441786-9bec-4201-9aae-83a19e97ccdd-kube-api-access-dbzzx\") pod \"e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") " pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:57.062820 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:57.062725    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:20:57.199880 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:57.199854    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"]
Jul 26 18:20:57.202596 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:20:57.202568    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b441786_9bec_4201_9aae_83a19e97ccdd.slice/crio-10764d15cb7d686da7026ea24cf329c024e4bb3d1d0b9a1a25eb38fa6f0b620d WatchSource:0}: Error finding container 10764d15cb7d686da7026ea24cf329c024e4bb3d1d0b9a1a25eb38fa6f0b620d: Status 404 returned error can't find the container with id 10764d15cb7d686da7026ea24cf329c024e4bb3d1d0b9a1a25eb38fa6f0b620d
Jul 26 18:20:57.622091 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:57.622052    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6" event={"ID":"5b441786-9bec-4201-9aae-83a19e97ccdd","Type":"ContainerStarted","Data":"1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4"}
Jul 26 18:20:57.622091 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:57.622099    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6" event={"ID":"5b441786-9bec-4201-9aae-83a19e97ccdd","Type":"ContainerStarted","Data":"10764d15cb7d686da7026ea24cf329c024e4bb3d1d0b9a1a25eb38fa6f0b620d"}
Jul 26 18:20:57.657123 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:20:57.657062    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6" podStartSLOduration=1.657045348 podStartE2EDuration="1.657045348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:57.654585113 +0000 UTC m=+1411.844614866" watchObservedRunningTime="2026-07-26 18:20:57.657045348 +0000 UTC m=+1411.847075102"
Jul 26 18:21:01.227662 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:01.227626    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:21:07.062951 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:07.062849    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:21:07.062951 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:07.062912    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:21:07.075626 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:07.075599    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:21:07.680784 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:07.680753    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:21:12.235382 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:12.235346    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:21:45.012951 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:45.012911    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"]
Jul 26 18:21:46.709875 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:46.709838    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:21:57.210629 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:21:57.210595    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:22:00.032168 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.032126    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6" podUID="5b441786-9bec-4201-9aae-83a19e97ccdd" containerName="main" containerID="cri-o://1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4" gracePeriod=45
Jul 26 18:22:00.276587 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.276563    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:22:00.391665 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391567    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b441786-9bec-4201-9aae-83a19e97ccdd-tls-certs\") pod \"5b441786-9bec-4201-9aae-83a19e97ccdd\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") "
Jul 26 18:22:00.391665 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391613    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-home\") pod \"5b441786-9bec-4201-9aae-83a19e97ccdd\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") "
Jul 26 18:22:00.391917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391688    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-tmp-dir\") pod \"5b441786-9bec-4201-9aae-83a19e97ccdd\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") "
Jul 26 18:22:00.391917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391750    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-dshm\") pod \"5b441786-9bec-4201-9aae-83a19e97ccdd\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") "
Jul 26 18:22:00.391917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391779    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbzzx\" (UniqueName: \"kubernetes.io/projected/5b441786-9bec-4201-9aae-83a19e97ccdd-kube-api-access-dbzzx\") pod \"5b441786-9bec-4201-9aae-83a19e97ccdd\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") "
Jul 26 18:22:00.391917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391810    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-model-cache\") pod \"5b441786-9bec-4201-9aae-83a19e97ccdd\" (UID: \"5b441786-9bec-4201-9aae-83a19e97ccdd\") "
Jul 26 18:22:00.392140 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391976    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-home" (OuterVolumeSpecName: "home") pod "5b441786-9bec-4201-9aae-83a19e97ccdd" (UID: "5b441786-9bec-4201-9aae-83a19e97ccdd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:22:00.392140 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.391993    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b441786-9bec-4201-9aae-83a19e97ccdd" (UID: "5b441786-9bec-4201-9aae-83a19e97ccdd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:22:00.392240 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.392174    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b441786-9bec-4201-9aae-83a19e97ccdd" (UID: "5b441786-9bec-4201-9aae-83a19e97ccdd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:22:00.392363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.392344    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:22:00.392434 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.392367    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:22:00.392434 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.392382    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:22:00.394011 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.393976    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-dshm" (OuterVolumeSpecName: "dshm") pod "5b441786-9bec-4201-9aae-83a19e97ccdd" (UID: "5b441786-9bec-4201-9aae-83a19e97ccdd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:22:00.394124 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.394037    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b441786-9bec-4201-9aae-83a19e97ccdd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b441786-9bec-4201-9aae-83a19e97ccdd" (UID: "5b441786-9bec-4201-9aae-83a19e97ccdd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:22:00.394124 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.394051    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b441786-9bec-4201-9aae-83a19e97ccdd-kube-api-access-dbzzx" (OuterVolumeSpecName: "kube-api-access-dbzzx") pod "5b441786-9bec-4201-9aae-83a19e97ccdd" (UID: "5b441786-9bec-4201-9aae-83a19e97ccdd"). InnerVolumeSpecName "kube-api-access-dbzzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:22:00.493306 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.493263    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b441786-9bec-4201-9aae-83a19e97ccdd-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:22:00.493306 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.493295    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbzzx\" (UniqueName: \"kubernetes.io/projected/5b441786-9bec-4201-9aae-83a19e97ccdd-kube-api-access-dbzzx\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:22:00.493306 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.493307    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b441786-9bec-4201-9aae-83a19e97ccdd-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:22:00.921080 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.921046    2565 generic.go:356] "Generic (PLEG): container finished" podID="5b441786-9bec-4201-9aae-83a19e97ccdd" containerID="1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4" exitCode=0
Jul 26 18:22:00.921080 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.921089    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"
Jul 26 18:22:00.921334 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.921122    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6" event={"ID":"5b441786-9bec-4201-9aae-83a19e97ccdd","Type":"ContainerDied","Data":"1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4"}
Jul 26 18:22:00.921334 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.921175    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6" event={"ID":"5b441786-9bec-4201-9aae-83a19e97ccdd","Type":"ContainerDied","Data":"10764d15cb7d686da7026ea24cf329c024e4bb3d1d0b9a1a25eb38fa6f0b620d"}
Jul 26 18:22:00.921334 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.921192    2565 scope.go:117] "RemoveContainer" containerID="1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4"
Jul 26 18:22:00.930884 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.930864    2565 scope.go:117] "RemoveContainer" containerID="1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4"
Jul 26 18:22:00.931190 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:22:00.931170    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4\": container with ID starting with 1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4 not found: ID does not exist" containerID="1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4"
Jul 26 18:22:00.931275 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.931201    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4"} err="failed to get container status \"1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4\": rpc error: code = NotFound desc = could not find container \"1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4\": container with ID starting with 1a5e7e60f9033fc7cd5c115249df4f97d34d44fdab07c2d5260876bbc3cfd1a4 not found: ID does not exist"
Jul 26 18:22:00.945205 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.945177    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"]
Jul 26 18:22:00.950074 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:00.950050    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-fea2be60-kserve-55f865467-4lmw6"]
Jul 26 18:22:02.318390 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:02.318361    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b441786-9bec-4201-9aae-83a19e97ccdd" path="/var/lib/kubelet/pods/5b441786-9bec-4201-9aae-83a19e97ccdd/volumes"
Jul 26 18:22:09.538267 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:09.538236    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:22:25.416321 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:25.416285    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:22:36.623237 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:36.623204    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:22:45.812981 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:45.812944    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:22:56.124783 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:22:56.124685    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:23:05.811894 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:23:05.811859    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:23:15.120584 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:23:15.120548    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:23:25.920690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:23:25.920653    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:23:37.621627 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:23:37.621593    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:23:46.021867 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:23:46.021833    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:23:56.017541 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:23:56.017507    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:24:15.817468 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:24:15.817430    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:24:28.113308 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:24:28.113230    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:24:39.109667 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:24:39.109631    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:24:51.513243 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:24:51.513212    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:24:59.514643 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:24:59.514609    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:25:09.627863 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:09.627807    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:25:22.515571 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:22.515533    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:25:38.516014 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:38.515978    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:25:56.125636 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.125553    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"]
Jul 26 18:25:56.126075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.125887    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b441786-9bec-4201-9aae-83a19e97ccdd" containerName="main"
Jul 26 18:25:56.126075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.125899    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b441786-9bec-4201-9aae-83a19e97ccdd" containerName="main"
Jul 26 18:25:56.126075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.125953    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b441786-9bec-4201-9aae-83a19e97ccdd" containerName="main"
Jul 26 18:25:56.128996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.128976    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:25:56.132148 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.132128    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:25:56.132487 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.132471    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:25:56.142803 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.142781    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"]
Jul 26 18:25:56.181007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.180962    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94c971b-4402-4a3a-82ad-980ea27381c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx\" (UID: \"f94c971b-4402-4a3a-82ad-980ea27381c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:25:56.281747 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.281688    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94c971b-4402-4a3a-82ad-980ea27381c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx\" (UID: \"f94c971b-4402-4a3a-82ad-980ea27381c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:25:56.282309 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.282287    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94c971b-4402-4a3a-82ad-980ea27381c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx\" (UID: \"f94c971b-4402-4a3a-82ad-980ea27381c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:25:56.438650 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.438574    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:25:56.571157 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.571121    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"]
Jul 26 18:25:56.575674 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:25:56.575646    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94c971b_4402_4a3a_82ad_980ea27381c3.slice/crio-bc990f2d799c970d9f968b7bc43645e4fffcdbf50e7b57c63bca500ee4ce8216 WatchSource:0}: Error finding container bc990f2d799c970d9f968b7bc43645e4fffcdbf50e7b57c63bca500ee4ce8216: Status 404 returned error can't find the container with id bc990f2d799c970d9f968b7bc43645e4fffcdbf50e7b57c63bca500ee4ce8216
Jul 26 18:25:56.577662 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:56.577645    2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:25:57.037588 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:25:57.037552    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx" event={"ID":"f94c971b-4402-4a3a-82ad-980ea27381c3","Type":"ContainerStarted","Data":"bc990f2d799c970d9f968b7bc43645e4fffcdbf50e7b57c63bca500ee4ce8216"}
Jul 26 18:26:00.055025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:00.054986    2565 generic.go:356] "Generic (PLEG): container finished" podID="f94c971b-4402-4a3a-82ad-980ea27381c3" containerID="0119645a1e914a4f169d6562e863b90f42d673afea2aec79554ab14f89bc0a7f" exitCode=0
Jul 26 18:26:00.055407 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:00.055047    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx" event={"ID":"f94c971b-4402-4a3a-82ad-980ea27381c3","Type":"ContainerDied","Data":"0119645a1e914a4f169d6562e863b90f42d673afea2aec79554ab14f89bc0a7f"}
Jul 26 18:26:01.184447 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:01.184423    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:26:01.222521 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:01.222486    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94c971b-4402-4a3a-82ad-980ea27381c3-maas-api-service-ca\") pod \"f94c971b-4402-4a3a-82ad-980ea27381c3\" (UID: \"f94c971b-4402-4a3a-82ad-980ea27381c3\") "
Jul 26 18:26:01.222846 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:01.222825    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f94c971b-4402-4a3a-82ad-980ea27381c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f94c971b-4402-4a3a-82ad-980ea27381c3" (UID: "f94c971b-4402-4a3a-82ad-980ea27381c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:26:01.323408 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:01.323319    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f94c971b-4402-4a3a-82ad-980ea27381c3-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:26:02.064748 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:02.064719    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"
Jul 26 18:26:02.064924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:02.064716    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx" event={"ID":"f94c971b-4402-4a3a-82ad-980ea27381c3","Type":"ContainerDied","Data":"bc990f2d799c970d9f968b7bc43645e4fffcdbf50e7b57c63bca500ee4ce8216"}
Jul 26 18:26:02.064924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:02.064837    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc990f2d799c970d9f968b7bc43645e4fffcdbf50e7b57c63bca500ee4ce8216"
Jul 26 18:26:09.617067 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:09.617029    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:26:11.225737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:11.225693    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"]
Jul 26 18:26:11.229982 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:11.229958    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c1daa682-rsvbx"]
Jul 26 18:26:12.318016 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:12.317976    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f94c971b-4402-4a3a-82ad-980ea27381c3" path="/var/lib/kubelet/pods/f94c971b-4402-4a3a-82ad-980ea27381c3/volumes"
Jul 26 18:26:15.889593 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.889558    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"]
Jul 26 18:26:15.889968 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.889888    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f94c971b-4402-4a3a-82ad-980ea27381c3" containerName="revoke-keys"
Jul 26 18:26:15.889968 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.889899    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f94c971b-4402-4a3a-82ad-980ea27381c3" containerName="revoke-keys"
Jul 26 18:26:15.889968 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.889951    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f94c971b-4402-4a3a-82ad-980ea27381c3" containerName="revoke-keys"
Jul 26 18:26:15.892974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.892957    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:15.895882 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.895842    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b6610536\""
Jul 26 18:26:15.896000 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.895980    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b6610536-dockercfg-ns4jk\""
Jul 26 18:26:15.903174 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.903154    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"]
Jul 26 18:26:15.938311 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.938276    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbd26\" (UniqueName: \"kubernetes.io/projected/841f52ba-830a-4e17-b23b-dcc876ee3162-kube-api-access-vbd26\") pod \"payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:15.938469 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:15.938318    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841f52ba-830a-4e17-b23b-dcc876ee3162-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:16.038885 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:16.038845    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841f52ba-830a-4e17-b23b-dcc876ee3162-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:16.039065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:16.038935    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbd26\" (UniqueName: \"kubernetes.io/projected/841f52ba-830a-4e17-b23b-dcc876ee3162-kube-api-access-vbd26\") pod \"payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:16.039606 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:16.039585    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841f52ba-830a-4e17-b23b-dcc876ee3162-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:16.047785 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:16.047761    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbd26\" (UniqueName: \"kubernetes.io/projected/841f52ba-830a-4e17-b23b-dcc876ee3162-kube-api-access-vbd26\") pod \"payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") " pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:16.203626 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:16.203548    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:16.332405 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:16.332380    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"]
Jul 26 18:26:16.334079 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:26:16.334052    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod841f52ba_830a_4e17_b23b_dcc876ee3162.slice/crio-84a9d90ec4eb828314d47b0e264d22c52362a64eaa3a40a748c09704554a5799 WatchSource:0}: Error finding container 84a9d90ec4eb828314d47b0e264d22c52362a64eaa3a40a748c09704554a5799: Status 404 returned error can't find the container with id 84a9d90ec4eb828314d47b0e264d22c52362a64eaa3a40a748c09704554a5799
Jul 26 18:26:17.144563 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:17.144482    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n" event={"ID":"841f52ba-830a-4e17-b23b-dcc876ee3162","Type":"ContainerStarted","Data":"84a9d90ec4eb828314d47b0e264d22c52362a64eaa3a40a748c09704554a5799"}
Jul 26 18:26:18.150886 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:18.150851    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n" event={"ID":"841f52ba-830a-4e17-b23b-dcc876ee3162","Type":"ContainerStarted","Data":"972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a"}
Jul 26 18:26:18.151405 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:18.150948    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:18.169445 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:18.169398    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n" podStartSLOduration=1.796178716 podStartE2EDuration="3.169382381s" totalImagesPullingTime="1.373203665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:26:18.167944171 +0000 UTC m=+1732.357973935" watchObservedRunningTime="2026-07-26 18:26:18.169382381 +0000 UTC m=+1732.359412135"
Jul 26 18:26:21.115232 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:21.115197    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:26:29.157884 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:29.157852    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:39.956748 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:39.956713    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"]
Jul 26 18:26:39.964925 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:39.964904    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:39.968308 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:39.968264    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:26:39.969162 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:39.968665    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:26:39.972326 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:39.971217    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"]
Jul 26 18:26:40.138119 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:40.138061    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f6b049b-371a-4f96-8078-c2fb976713ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b6610536-f729c\" (UID: \"0f6b049b-371a-4f96-8078-c2fb976713ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:40.238692 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:40.238652    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f6b049b-371a-4f96-8078-c2fb976713ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b6610536-f729c\" (UID: \"0f6b049b-371a-4f96-8078-c2fb976713ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:40.239347 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:40.239325    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f6b049b-371a-4f96-8078-c2fb976713ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b6610536-f729c\" (UID: \"0f6b049b-371a-4f96-8078-c2fb976713ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:40.279913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:40.279864    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:40.409455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:40.409429    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"]
Jul 26 18:26:40.411522 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:26:40.411493    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6b049b_371a_4f96_8078_c2fb976713ef.slice/crio-26d3bc8cd209bfcc7651e34f1bb8698abf200b04e7b784850fffd28a41944af6 WatchSource:0}: Error finding container 26d3bc8cd209bfcc7651e34f1bb8698abf200b04e7b784850fffd28a41944af6: Status 404 returned error can't find the container with id 26d3bc8cd209bfcc7651e34f1bb8698abf200b04e7b784850fffd28a41944af6
Jul 26 18:26:41.264343 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:41.264259    2565 generic.go:356] "Generic (PLEG): container finished" podID="0f6b049b-371a-4f96-8078-c2fb976713ef" containerID="86be99aacf9934e563898dd3998e1a26c04848f3ff97ecf344cc18f7d0eb401c" exitCode=0
Jul 26 18:26:41.264712 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:41.264346    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c" event={"ID":"0f6b049b-371a-4f96-8078-c2fb976713ef","Type":"ContainerDied","Data":"86be99aacf9934e563898dd3998e1a26c04848f3ff97ecf344cc18f7d0eb401c"}
Jul 26 18:26:41.264712 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:41.264399    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c" event={"ID":"0f6b049b-371a-4f96-8078-c2fb976713ef","Type":"ContainerStarted","Data":"26d3bc8cd209bfcc7651e34f1bb8698abf200b04e7b784850fffd28a41944af6"}
Jul 26 18:26:42.400333 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:42.400308    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:42.557530 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:42.557444    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f6b049b-371a-4f96-8078-c2fb976713ef-maas-api-service-ca\") pod \"0f6b049b-371a-4f96-8078-c2fb976713ef\" (UID: \"0f6b049b-371a-4f96-8078-c2fb976713ef\") "
Jul 26 18:26:42.557888 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:42.557855    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f6b049b-371a-4f96-8078-c2fb976713ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0f6b049b-371a-4f96-8078-c2fb976713ef" (UID: "0f6b049b-371a-4f96-8078-c2fb976713ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:26:42.658804 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:42.658776    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f6b049b-371a-4f96-8078-c2fb976713ef-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:26:43.275669 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:43.275637    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c" event={"ID":"0f6b049b-371a-4f96-8078-c2fb976713ef","Type":"ContainerDied","Data":"26d3bc8cd209bfcc7651e34f1bb8698abf200b04e7b784850fffd28a41944af6"}
Jul 26 18:26:43.275669 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:43.275656    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"
Jul 26 18:26:43.275941 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:43.275686    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26d3bc8cd209bfcc7651e34f1bb8698abf200b04e7b784850fffd28a41944af6"
Jul 26 18:26:50.461481 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.461446    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"]
Jul 26 18:26:50.461992 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.461667    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n" podUID="841f52ba-830a-4e17-b23b-dcc876ee3162" containerName="payload-processing" containerID="cri-o://972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a" gracePeriod=30
Jul 26 18:26:50.725113 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.725094    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:50.831282 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.831241    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbd26\" (UniqueName: \"kubernetes.io/projected/841f52ba-830a-4e17-b23b-dcc876ee3162-kube-api-access-vbd26\") pod \"841f52ba-830a-4e17-b23b-dcc876ee3162\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") "
Jul 26 18:26:50.831472 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.831326    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841f52ba-830a-4e17-b23b-dcc876ee3162-plugins-config-volume\") pod \"841f52ba-830a-4e17-b23b-dcc876ee3162\" (UID: \"841f52ba-830a-4e17-b23b-dcc876ee3162\") "
Jul 26 18:26:50.831692 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.831662    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/841f52ba-830a-4e17-b23b-dcc876ee3162-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "841f52ba-830a-4e17-b23b-dcc876ee3162" (UID: "841f52ba-830a-4e17-b23b-dcc876ee3162"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:26:50.833543 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.833506    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/841f52ba-830a-4e17-b23b-dcc876ee3162-kube-api-access-vbd26" (OuterVolumeSpecName: "kube-api-access-vbd26") pod "841f52ba-830a-4e17-b23b-dcc876ee3162" (UID: "841f52ba-830a-4e17-b23b-dcc876ee3162"). InnerVolumeSpecName "kube-api-access-vbd26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:26:50.932737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.932686    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/841f52ba-830a-4e17-b23b-dcc876ee3162-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:26:50.932737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:50.932731    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbd26\" (UniqueName: \"kubernetes.io/projected/841f52ba-830a-4e17-b23b-dcc876ee3162-kube-api-access-vbd26\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:26:51.325249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.325213    2565 generic.go:356] "Generic (PLEG): container finished" podID="841f52ba-830a-4e17-b23b-dcc876ee3162" containerID="972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a" exitCode=0
Jul 26 18:26:51.325444 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.325269    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"
Jul 26 18:26:51.325444 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.325282    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n" event={"ID":"841f52ba-830a-4e17-b23b-dcc876ee3162","Type":"ContainerDied","Data":"972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a"}
Jul 26 18:26:51.325444 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.325329    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n" event={"ID":"841f52ba-830a-4e17-b23b-dcc876ee3162","Type":"ContainerDied","Data":"84a9d90ec4eb828314d47b0e264d22c52362a64eaa3a40a748c09704554a5799"}
Jul 26 18:26:51.325444 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.325361    2565 scope.go:117] "RemoveContainer" containerID="972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a"
Jul 26 18:26:51.334103 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.334084    2565 scope.go:117] "RemoveContainer" containerID="972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a"
Jul 26 18:26:51.334327 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:26:51.334311    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a\": container with ID starting with 972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a not found: ID does not exist" containerID="972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a"
Jul 26 18:26:51.334368 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.334334    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a"} err="failed to get container status \"972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a\": rpc error: code = NotFound desc = could not find container \"972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a\": container with ID starting with 972c2a7d095a52389234c81b876122d8b356c43554f4149097e76091ae7e2c7a not found: ID does not exist"
Jul 26 18:26:51.346766 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.346741    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"]
Jul 26 18:26:51.353240 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:51.353218    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b6610536-85579b7bf7-rbz7n"]
Jul 26 18:26:52.319893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:52.319861    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="841f52ba-830a-4e17-b23b-dcc876ee3162" path="/var/lib/kubelet/pods/841f52ba-830a-4e17-b23b-dcc876ee3162/volumes"
Jul 26 18:26:53.518412 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:53.518375    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:26:55.041359 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:55.041326    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"]
Jul 26 18:26:55.044714 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:55.044665    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b6610536-f729c"]
Jul 26 18:26:56.318648 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:56.318614    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f6b049b-371a-4f96-8078-c2fb976713ef" path="/var/lib/kubelet/pods/0f6b049b-371a-4f96-8078-c2fb976713ef/volumes"
Jul 26 18:26:58.943595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943561    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"]
Jul 26 18:26:58.943998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943857    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="841f52ba-830a-4e17-b23b-dcc876ee3162" containerName="payload-processing"
Jul 26 18:26:58.943998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943868    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="841f52ba-830a-4e17-b23b-dcc876ee3162" containerName="payload-processing"
Jul 26 18:26:58.943998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943876    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f6b049b-371a-4f96-8078-c2fb976713ef" containerName="revoke-keys"
Jul 26 18:26:58.943998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943882    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f6b049b-371a-4f96-8078-c2fb976713ef" containerName="revoke-keys"
Jul 26 18:26:58.943998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943943    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f6b049b-371a-4f96-8078-c2fb976713ef" containerName="revoke-keys"
Jul 26 18:26:58.943998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.943951    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="841f52ba-830a-4e17-b23b-dcc876ee3162" containerName="payload-processing"
Jul 26 18:26:58.946732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.946716    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:58.949311 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.949291    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-60637c1e\""
Jul 26 18:26:58.949311 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.949299    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-60637c1e-dockercfg-dsvqh\""
Jul 26 18:26:58.956085 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.956065    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"]
Jul 26 18:26:58.989224 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.989192    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e096c92-d156-49ce-8b91-4c782b35463a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:58.989224 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:58.989226    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgvql\" (UniqueName: \"kubernetes.io/projected/5e096c92-d156-49ce-8b91-4c782b35463a-kube-api-access-vgvql\") pod \"payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:59.089775 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.089734    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e096c92-d156-49ce-8b91-4c782b35463a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:59.089775 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.089779    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgvql\" (UniqueName: \"kubernetes.io/projected/5e096c92-d156-49ce-8b91-4c782b35463a-kube-api-access-vgvql\") pod \"payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:59.090417 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.090399    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e096c92-d156-49ce-8b91-4c782b35463a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:59.100160 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.100132    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"]
Jul 26 18:26:59.102339 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.102315    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgvql\" (UniqueName: \"kubernetes.io/projected/5e096c92-d156-49ce-8b91-4c782b35463a-kube-api-access-vgvql\") pod \"payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:59.103425 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.103411    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.114155 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.114129    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"]
Jul 26 18:26:59.191130 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.191095    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/664da633-f940-436d-88ca-97aa088218da-plugins-config-volume\") pod \"payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") " pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.191130 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.191137    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtghm\" (UniqueName: \"kubernetes.io/projected/664da633-f940-436d-88ca-97aa088218da-kube-api-access-dtghm\") pod \"payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") " pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.257025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.256990    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:26:59.292327 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.292293    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/664da633-f940-436d-88ca-97aa088218da-plugins-config-volume\") pod \"payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") " pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.292477 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.292336    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtghm\" (UniqueName: \"kubernetes.io/projected/664da633-f940-436d-88ca-97aa088218da-kube-api-access-dtghm\") pod \"payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") " pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.293084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.293061    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/664da633-f940-436d-88ca-97aa088218da-plugins-config-volume\") pod \"payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") " pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.301348 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.301312    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtghm\" (UniqueName: \"kubernetes.io/projected/664da633-f940-436d-88ca-97aa088218da-kube-api-access-dtghm\") pod \"payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") " pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.397959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.397934    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"]
Jul 26 18:26:59.399849 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:26:59.399822    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e096c92_d156_49ce_8b91_4c782b35463a.slice/crio-50adfd1eca226bcb998d03dbab7e5909bfb18afaecd879a1f1da6417352eb0fd WatchSource:0}: Error finding container 50adfd1eca226bcb998d03dbab7e5909bfb18afaecd879a1f1da6417352eb0fd: Status 404 returned error can't find the container with id 50adfd1eca226bcb998d03dbab7e5909bfb18afaecd879a1f1da6417352eb0fd
Jul 26 18:26:59.419101 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.419077    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:26:59.554042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:26:59.553972    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"]
Jul 26 18:26:59.559130 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:26:59.559100    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664da633_f940_436d_88ca_97aa088218da.slice/crio-29ea91de598f6d62e052de2dd18b8457f49e4b0845a168942b6a7f86be79da27 WatchSource:0}: Error finding container 29ea91de598f6d62e052de2dd18b8457f49e4b0845a168942b6a7f86be79da27: Status 404 returned error can't find the container with id 29ea91de598f6d62e052de2dd18b8457f49e4b0845a168942b6a7f86be79da27
Jul 26 18:27:00.371425 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.371386    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4" event={"ID":"5e096c92-d156-49ce-8b91-4c782b35463a","Type":"ContainerStarted","Data":"68a60593e060554caae0b5840f1f5f3259565d6a6d02226edb98e4f32af9fcbf"}
Jul 26 18:27:00.372023 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.371451    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4" event={"ID":"5e096c92-d156-49ce-8b91-4c782b35463a","Type":"ContainerStarted","Data":"50adfd1eca226bcb998d03dbab7e5909bfb18afaecd879a1f1da6417352eb0fd"}
Jul 26 18:27:00.372023 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.371483    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:27:00.373054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.373026    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7" event={"ID":"664da633-f940-436d-88ca-97aa088218da","Type":"ContainerStarted","Data":"855cbb1b58173ba2c130f892b706262f1972bf66aead762b703610c647a3e43b"}
Jul 26 18:27:00.373168 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.373078    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7" event={"ID":"664da633-f940-436d-88ca-97aa088218da","Type":"ContainerStarted","Data":"29ea91de598f6d62e052de2dd18b8457f49e4b0845a168942b6a7f86be79da27"}
Jul 26 18:27:00.373168 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.373106    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:27:00.398120 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:00.398064    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4" podStartSLOduration=2.398047874 podStartE2EDuration="2.398047874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:27:00.397234078 +0000 UTC m=+1774.587263834" watchObservedRunningTime="2026-07-26 18:27:00.398047874 +0000 UTC m=+1774.588077627"
Jul 26 18:27:05.417138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:05.417084    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7" podStartSLOduration=6.4170663789999995 podStartE2EDuration="6.417066379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:26:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:27:00.424081912 +0000 UTC m=+1774.614111666" watchObservedRunningTime="2026-07-26 18:27:05.417066379 +0000 UTC m=+1779.607096132"
Jul 26 18:27:05.418504 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:05.418485    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:27:11.380244 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:11.380213    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:27:11.380683 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:11.380293    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:27:23.862992 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:23.862966    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"]
Jul 26 18:27:23.866315 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:23.866296    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:23.868857 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:23.868820    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:27:23.868973 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:23.868930    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:27:23.873737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:23.873651    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"]
Jul 26 18:27:23.981926 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:23.981895    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c470fd-3a3f-482e-a66a-70b78aa24743-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60637c1e-57kzt\" (UID: \"a5c470fd-3a3f-482e-a66a-70b78aa24743\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:24.082807 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:24.082754    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c470fd-3a3f-482e-a66a-70b78aa24743-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60637c1e-57kzt\" (UID: \"a5c470fd-3a3f-482e-a66a-70b78aa24743\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:24.083411 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:24.083391    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c470fd-3a3f-482e-a66a-70b78aa24743-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-60637c1e-57kzt\" (UID: \"a5c470fd-3a3f-482e-a66a-70b78aa24743\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:24.178993 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:24.178907    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:24.318788 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:27:24.318760    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5c470fd_3a3f_482e_a66a_70b78aa24743.slice/crio-104b7184618fb27b6d1d0e905561c15aa7937a393ffe315f703121cbdaa33e02 WatchSource:0}: Error finding container 104b7184618fb27b6d1d0e905561c15aa7937a393ffe315f703121cbdaa33e02: Status 404 returned error can't find the container with id 104b7184618fb27b6d1d0e905561c15aa7937a393ffe315f703121cbdaa33e02
Jul 26 18:27:24.325066 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:24.322004    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"]
Jul 26 18:27:24.493116 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:24.493084    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt" event={"ID":"a5c470fd-3a3f-482e-a66a-70b78aa24743","Type":"ContainerStarted","Data":"104b7184618fb27b6d1d0e905561c15aa7937a393ffe315f703121cbdaa33e02"}
Jul 26 18:27:25.500172 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:25.500133    2565 generic.go:356] "Generic (PLEG): container finished" podID="a5c470fd-3a3f-482e-a66a-70b78aa24743" containerID="a5b187ff999d20dfadf5166a8e1f4d6c9cc2c42b6651a3bb1f9324363d3faff5" exitCode=0
Jul 26 18:27:25.500551 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:25.500214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt" event={"ID":"a5c470fd-3a3f-482e-a66a-70b78aa24743","Type":"ContainerDied","Data":"a5b187ff999d20dfadf5166a8e1f4d6c9cc2c42b6651a3bb1f9324363d3faff5"}
Jul 26 18:27:26.631905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:26.631880    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:26.708856 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:26.708826    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c470fd-3a3f-482e-a66a-70b78aa24743-maas-api-service-ca\") pod \"a5c470fd-3a3f-482e-a66a-70b78aa24743\" (UID: \"a5c470fd-3a3f-482e-a66a-70b78aa24743\") "
Jul 26 18:27:26.709168 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:26.709146    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5c470fd-3a3f-482e-a66a-70b78aa24743-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a5c470fd-3a3f-482e-a66a-70b78aa24743" (UID: "a5c470fd-3a3f-482e-a66a-70b78aa24743"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:26.810309 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:26.810281    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5c470fd-3a3f-482e-a66a-70b78aa24743-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:27:27.511048 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:27.511021    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"
Jul 26 18:27:27.511048 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:27.511029    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt" event={"ID":"a5c470fd-3a3f-482e-a66a-70b78aa24743","Type":"ContainerDied","Data":"104b7184618fb27b6d1d0e905561c15aa7937a393ffe315f703121cbdaa33e02"}
Jul 26 18:27:27.511249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:27.511071    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="104b7184618fb27b6d1d0e905561c15aa7937a393ffe315f703121cbdaa33e02"
Jul 26 18:27:34.329540 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.329107    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"]
Jul 26 18:27:34.330485 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.330217    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7" podUID="664da633-f940-436d-88ca-97aa088218da" containerName="payload-processing" containerID="cri-o://855cbb1b58173ba2c130f892b706262f1972bf66aead762b703610c647a3e43b" gracePeriod=30
Jul 26 18:27:34.332149 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.332125    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"]
Jul 26 18:27:34.332394 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.332355    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4" podUID="5e096c92-d156-49ce-8b91-4c782b35463a" containerName="payload-pre-processing" containerID="cri-o://68a60593e060554caae0b5840f1f5f3259565d6a6d02226edb98e4f32af9fcbf" gracePeriod=30
Jul 26 18:27:34.547268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.547054    2565 generic.go:356] "Generic (PLEG): container finished" podID="664da633-f940-436d-88ca-97aa088218da" containerID="855cbb1b58173ba2c130f892b706262f1972bf66aead762b703610c647a3e43b" exitCode=0
Jul 26 18:27:34.547268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.547134    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7" event={"ID":"664da633-f940-436d-88ca-97aa088218da","Type":"ContainerDied","Data":"855cbb1b58173ba2c130f892b706262f1972bf66aead762b703610c647a3e43b"}
Jul 26 18:27:34.549806 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.549775    2565 generic.go:356] "Generic (PLEG): container finished" podID="5e096c92-d156-49ce-8b91-4c782b35463a" containerID="68a60593e060554caae0b5840f1f5f3259565d6a6d02226edb98e4f32af9fcbf" exitCode=0
Jul 26 18:27:34.549934 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.549846    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4" event={"ID":"5e096c92-d156-49ce-8b91-4c782b35463a","Type":"ContainerDied","Data":"68a60593e060554caae0b5840f1f5f3259565d6a6d02226edb98e4f32af9fcbf"}
Jul 26 18:27:34.594031 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.594006    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:27:34.620971 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.620948    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:27:34.673442 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.673415    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/664da633-f940-436d-88ca-97aa088218da-plugins-config-volume\") pod \"664da633-f940-436d-88ca-97aa088218da\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") "
Jul 26 18:27:34.673596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.673451    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgvql\" (UniqueName: \"kubernetes.io/projected/5e096c92-d156-49ce-8b91-4c782b35463a-kube-api-access-vgvql\") pod \"5e096c92-d156-49ce-8b91-4c782b35463a\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") "
Jul 26 18:27:34.673596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.673494    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e096c92-d156-49ce-8b91-4c782b35463a-plugins-config-volume\") pod \"5e096c92-d156-49ce-8b91-4c782b35463a\" (UID: \"5e096c92-d156-49ce-8b91-4c782b35463a\") "
Jul 26 18:27:34.673596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.673531    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtghm\" (UniqueName: \"kubernetes.io/projected/664da633-f940-436d-88ca-97aa088218da-kube-api-access-dtghm\") pod \"664da633-f940-436d-88ca-97aa088218da\" (UID: \"664da633-f940-436d-88ca-97aa088218da\") "
Jul 26 18:27:34.673902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.673862    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/664da633-f940-436d-88ca-97aa088218da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "664da633-f940-436d-88ca-97aa088218da" (UID: "664da633-f940-436d-88ca-97aa088218da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:34.673902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.673892    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e096c92-d156-49ce-8b91-4c782b35463a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e096c92-d156-49ce-8b91-4c782b35463a" (UID: "5e096c92-d156-49ce-8b91-4c782b35463a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:27:34.675618 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.675587    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e096c92-d156-49ce-8b91-4c782b35463a-kube-api-access-vgvql" (OuterVolumeSpecName: "kube-api-access-vgvql") pod "5e096c92-d156-49ce-8b91-4c782b35463a" (UID: "5e096c92-d156-49ce-8b91-4c782b35463a"). InnerVolumeSpecName "kube-api-access-vgvql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:34.675618 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.675600    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/664da633-f940-436d-88ca-97aa088218da-kube-api-access-dtghm" (OuterVolumeSpecName: "kube-api-access-dtghm") pod "664da633-f940-436d-88ca-97aa088218da" (UID: "664da633-f940-436d-88ca-97aa088218da"). InnerVolumeSpecName "kube-api-access-dtghm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:27:34.774764 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.774723    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/664da633-f940-436d-88ca-97aa088218da-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:27:34.774764 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.774757    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgvql\" (UniqueName: \"kubernetes.io/projected/5e096c92-d156-49ce-8b91-4c782b35463a-kube-api-access-vgvql\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:27:34.774764 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.774770    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e096c92-d156-49ce-8b91-4c782b35463a-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:27:34.774997 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:34.774782    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtghm\" (UniqueName: \"kubernetes.io/projected/664da633-f940-436d-88ca-97aa088218da-kube-api-access-dtghm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:27:35.556259 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.556230    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"
Jul 26 18:27:35.556745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.556282    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7" event={"ID":"664da633-f940-436d-88ca-97aa088218da","Type":"ContainerDied","Data":"29ea91de598f6d62e052de2dd18b8457f49e4b0845a168942b6a7f86be79da27"}
Jul 26 18:27:35.556745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.556359    2565 scope.go:117] "RemoveContainer" containerID="855cbb1b58173ba2c130f892b706262f1972bf66aead762b703610c647a3e43b"
Jul 26 18:27:35.557724 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.557593    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4" event={"ID":"5e096c92-d156-49ce-8b91-4c782b35463a","Type":"ContainerDied","Data":"50adfd1eca226bcb998d03dbab7e5909bfb18afaecd879a1f1da6417352eb0fd"}
Jul 26 18:27:35.557724 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.557617    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"
Jul 26 18:27:35.565293 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.565171    2565 scope.go:117] "RemoveContainer" containerID="68a60593e060554caae0b5840f1f5f3259565d6a6d02226edb98e4f32af9fcbf"
Jul 26 18:27:35.580542 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.580518    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"]
Jul 26 18:27:35.584894 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.584873    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-60637c1e-7dd64f656-vlns4"]
Jul 26 18:27:35.595204 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.595160    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"]
Jul 26 18:27:35.597095 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:35.597043    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-60637c1e-85f8455c66-xrss7"]
Jul 26 18:27:36.319684 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:36.319653    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e096c92-d156-49ce-8b91-4c782b35463a" path="/var/lib/kubelet/pods/5e096c92-d156-49ce-8b91-4c782b35463a/volumes"
Jul 26 18:27:36.320014 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:36.320001    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="664da633-f940-436d-88ca-97aa088218da" path="/var/lib/kubelet/pods/664da633-f940-436d-88ca-97aa088218da/volumes"
Jul 26 18:27:37.314983 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:37.314947    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:27:38.941573 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:38.941534    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"]
Jul 26 18:27:38.946514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:38.946490    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-60637c1e-57kzt"]
Jul 26 18:27:40.319092 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:40.319057    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5c470fd-3a3f-482e-a66a-70b78aa24743" path="/var/lib/kubelet/pods/a5c470fd-3a3f-482e-a66a-70b78aa24743/volumes"
Jul 26 18:27:44.280913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.280878    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"]
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281172    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e096c92-d156-49ce-8b91-4c782b35463a" containerName="payload-pre-processing"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281183    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e096c92-d156-49ce-8b91-4c782b35463a" containerName="payload-pre-processing"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281197    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="664da633-f940-436d-88ca-97aa088218da" containerName="payload-processing"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281203    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="664da633-f940-436d-88ca-97aa088218da" containerName="payload-processing"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281211    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5c470fd-3a3f-482e-a66a-70b78aa24743" containerName="revoke-keys"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281216    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5c470fd-3a3f-482e-a66a-70b78aa24743" containerName="revoke-keys"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281270    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5c470fd-3a3f-482e-a66a-70b78aa24743" containerName="revoke-keys"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281278    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e096c92-d156-49ce-8b91-4c782b35463a" containerName="payload-pre-processing"
Jul 26 18:27:44.281313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.281283    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="664da633-f940-436d-88ca-97aa088218da" containerName="payload-processing"
Jul 26 18:27:44.285381 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.285366    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.288323 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.288291    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ded9d893\""
Jul 26 18:27:44.288419 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.288325    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ded9d893-dockercfg-8f2g6\""
Jul 26 18:27:44.293399 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.293379    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"]
Jul 26 18:27:44.323676 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.323637    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"]
Jul 26 18:27:44.327829 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.327807    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.341605 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.341577    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"]
Jul 26 18:27:44.357243 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.357217    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrkbl\" (UniqueName: \"kubernetes.io/projected/7b24dda0-4030-406b-8b84-85d8b7343d52-kube-api-access-vrkbl\") pod \"payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.357383 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.357296    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b24dda0-4030-406b-8b84-85d8b7343d52-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.457934 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.457891    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d89146e6-db95-42e7-b963-5b39f3d1f60b-plugins-config-volume\") pod \"payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") " pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.458125 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.457957    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b24dda0-4030-406b-8b84-85d8b7343d52-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.458125 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.457989    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrkbl\" (UniqueName: \"kubernetes.io/projected/7b24dda0-4030-406b-8b84-85d8b7343d52-kube-api-access-vrkbl\") pod \"payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.458125 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.458014    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzz4g\" (UniqueName: \"kubernetes.io/projected/d89146e6-db95-42e7-b963-5b39f3d1f60b-kube-api-access-hzz4g\") pod \"payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") " pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.458782 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.458763    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b24dda0-4030-406b-8b84-85d8b7343d52-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.466401 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.466382    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrkbl\" (UniqueName: \"kubernetes.io/projected/7b24dda0-4030-406b-8b84-85d8b7343d52-kube-api-access-vrkbl\") pod \"payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.558848 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.558744    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d89146e6-db95-42e7-b963-5b39f3d1f60b-plugins-config-volume\") pod \"payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") " pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.558848 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.558813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzz4g\" (UniqueName: \"kubernetes.io/projected/d89146e6-db95-42e7-b963-5b39f3d1f60b-kube-api-access-hzz4g\") pod \"payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") " pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.559517 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.559496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d89146e6-db95-42e7-b963-5b39f3d1f60b-plugins-config-volume\") pod \"payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") " pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.567831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.567804    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzz4g\" (UniqueName: \"kubernetes.io/projected/d89146e6-db95-42e7-b963-5b39f3d1f60b-kube-api-access-hzz4g\") pod \"payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") " pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.597045 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.597019    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:44.637975 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.637946    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:44.739462 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.739394    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"]
Jul 26 18:27:44.742492 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:27:44.742462    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b24dda0_4030_406b_8b84_85d8b7343d52.slice/crio-90ffad34950c59aa72c015aebaa3631fe6fe3fb6629202ecd8fab37e8d977ab2 WatchSource:0}: Error finding container 90ffad34950c59aa72c015aebaa3631fe6fe3fb6629202ecd8fab37e8d977ab2: Status 404 returned error can't find the container with id 90ffad34950c59aa72c015aebaa3631fe6fe3fb6629202ecd8fab37e8d977ab2
Jul 26 18:27:44.789028 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:44.789002    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"]
Jul 26 18:27:44.790447 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:27:44.790426    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd89146e6_db95_42e7_b963_5b39f3d1f60b.slice/crio-4d3338f3515caeec35979a914513912e6aaa1eaf624f137fee243ceb6a3c1d58 WatchSource:0}: Error finding container 4d3338f3515caeec35979a914513912e6aaa1eaf624f137fee243ceb6a3c1d58: Status 404 returned error can't find the container with id 4d3338f3515caeec35979a914513912e6aaa1eaf624f137fee243ceb6a3c1d58
Jul 26 18:27:45.608388 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.608351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c" event={"ID":"d89146e6-db95-42e7-b963-5b39f3d1f60b","Type":"ContainerStarted","Data":"f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de"}
Jul 26 18:27:45.608872 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.608407    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c" event={"ID":"d89146e6-db95-42e7-b963-5b39f3d1f60b","Type":"ContainerStarted","Data":"4d3338f3515caeec35979a914513912e6aaa1eaf624f137fee243ceb6a3c1d58"}
Jul 26 18:27:45.608872 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.608455    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:45.610111 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.610080    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb" event={"ID":"7b24dda0-4030-406b-8b84-85d8b7343d52","Type":"ContainerStarted","Data":"782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af"}
Jul 26 18:27:45.610222 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.610127    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb" event={"ID":"7b24dda0-4030-406b-8b84-85d8b7343d52","Type":"ContainerStarted","Data":"90ffad34950c59aa72c015aebaa3631fe6fe3fb6629202ecd8fab37e8d977ab2"}
Jul 26 18:27:45.610222 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.610169    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:27:45.624875 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.624823    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c" podStartSLOduration=1.6248042040000001 podStartE2EDuration="1.624804204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:27:45.624419414 +0000 UTC m=+1819.814449172" watchObservedRunningTime="2026-07-26 18:27:45.624804204 +0000 UTC m=+1819.814833963"
Jul 26 18:27:45.642053 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:45.642002    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb" podStartSLOduration=1.6419855349999999 podStartE2EDuration="1.641985535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:27:45.640579908 +0000 UTC m=+1819.830609663" watchObservedRunningTime="2026-07-26 18:27:45.641985535 +0000 UTC m=+1819.832015288"
Jul 26 18:27:48.911779 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:48.911747    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:27:56.616789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:56.616761    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:27:56.617122 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:27:56.616831    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:28:08.220907 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.220864    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"]
Jul 26 18:28:08.224303 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.224280    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:08.226871 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.226850    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:28:08.227136 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.227120    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:28:08.233550 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.233527    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"]
Jul 26 18:28:08.249649 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.249617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59339196-1dfd-4487-b894-22aa26c6ec38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ded9d893-srj7k\" (UID: \"59339196-1dfd-4487-b894-22aa26c6ec38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:08.350275 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.350240    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59339196-1dfd-4487-b894-22aa26c6ec38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ded9d893-srj7k\" (UID: \"59339196-1dfd-4487-b894-22aa26c6ec38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:08.350959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.350932    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59339196-1dfd-4487-b894-22aa26c6ec38-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-ded9d893-srj7k\" (UID: \"59339196-1dfd-4487-b894-22aa26c6ec38\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:08.535636 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.535548    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:08.666656 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.666627    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"]
Jul 26 18:28:08.669785 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:28:08.669751    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59339196_1dfd_4487_b894_22aa26c6ec38.slice/crio-5a68e2f37d35111ce3674ef4133b885b78637c97335acf6c3a0bd75912e4ff06 WatchSource:0}: Error finding container 5a68e2f37d35111ce3674ef4133b885b78637c97335acf6c3a0bd75912e4ff06: Status 404 returned error can't find the container with id 5a68e2f37d35111ce3674ef4133b885b78637c97335acf6c3a0bd75912e4ff06
Jul 26 18:28:08.727879 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:08.727846    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k" event={"ID":"59339196-1dfd-4487-b894-22aa26c6ec38","Type":"ContainerStarted","Data":"5a68e2f37d35111ce3674ef4133b885b78637c97335acf6c3a0bd75912e4ff06"}
Jul 26 18:28:09.734364 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:09.734328    2565 generic.go:356] "Generic (PLEG): container finished" podID="59339196-1dfd-4487-b894-22aa26c6ec38" containerID="5670c5e27511d63349d67d7cc824073e7b6346917c1c2fc14394aebaa632a27a" exitCode=0
Jul 26 18:28:09.734817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:09.734409    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k" event={"ID":"59339196-1dfd-4487-b894-22aa26c6ec38","Type":"ContainerDied","Data":"5670c5e27511d63349d67d7cc824073e7b6346917c1c2fc14394aebaa632a27a"}
Jul 26 18:28:10.866056 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:10.866032    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:10.974455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:10.974405    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59339196-1dfd-4487-b894-22aa26c6ec38-maas-api-service-ca\") pod \"59339196-1dfd-4487-b894-22aa26c6ec38\" (UID: \"59339196-1dfd-4487-b894-22aa26c6ec38\") "
Jul 26 18:28:10.974831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:10.974809    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59339196-1dfd-4487-b894-22aa26c6ec38-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "59339196-1dfd-4487-b894-22aa26c6ec38" (UID: "59339196-1dfd-4487-b894-22aa26c6ec38"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:28:11.075541 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:11.075465    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/59339196-1dfd-4487-b894-22aa26c6ec38-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:28:11.745360 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:11.745329    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"
Jul 26 18:28:11.745360 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:11.745351    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k" event={"ID":"59339196-1dfd-4487-b894-22aa26c6ec38","Type":"ContainerDied","Data":"5a68e2f37d35111ce3674ef4133b885b78637c97335acf6c3a0bd75912e4ff06"}
Jul 26 18:28:11.745608 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:11.745392    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a68e2f37d35111ce3674ef4133b885b78637c97335acf6c3a0bd75912e4ff06"
Jul 26 18:28:18.693047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:18.693017    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"]
Jul 26 18:28:18.693463 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:18.693222    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c" podUID="d89146e6-db95-42e7-b963-5b39f3d1f60b" containerName="payload-processing" containerID="cri-o://f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de" gracePeriod=30
Jul 26 18:28:18.700582 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:18.700494    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"]
Jul 26 18:28:18.700772 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:18.700743    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb" podUID="7b24dda0-4030-406b-8b84-85d8b7343d52" containerName="payload-pre-processing" containerID="cri-o://782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af" gracePeriod=30
Jul 26 18:28:18.975096 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:18.975073    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:28:18.977105 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:18.977086    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:28:19.038802 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.038774    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d89146e6-db95-42e7-b963-5b39f3d1f60b-plugins-config-volume\") pod \"d89146e6-db95-42e7-b963-5b39f3d1f60b\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") "
Jul 26 18:28:19.038983 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.038843    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrkbl\" (UniqueName: \"kubernetes.io/projected/7b24dda0-4030-406b-8b84-85d8b7343d52-kube-api-access-vrkbl\") pod \"7b24dda0-4030-406b-8b84-85d8b7343d52\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") "
Jul 26 18:28:19.038983 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.038866    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b24dda0-4030-406b-8b84-85d8b7343d52-plugins-config-volume\") pod \"7b24dda0-4030-406b-8b84-85d8b7343d52\" (UID: \"7b24dda0-4030-406b-8b84-85d8b7343d52\") "
Jul 26 18:28:19.038983 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.038888    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzz4g\" (UniqueName: \"kubernetes.io/projected/d89146e6-db95-42e7-b963-5b39f3d1f60b-kube-api-access-hzz4g\") pod \"d89146e6-db95-42e7-b963-5b39f3d1f60b\" (UID: \"d89146e6-db95-42e7-b963-5b39f3d1f60b\") "
Jul 26 18:28:19.039259 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.039226    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d89146e6-db95-42e7-b963-5b39f3d1f60b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d89146e6-db95-42e7-b963-5b39f3d1f60b" (UID: "d89146e6-db95-42e7-b963-5b39f3d1f60b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:28:19.039259 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.039247    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b24dda0-4030-406b-8b84-85d8b7343d52-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b24dda0-4030-406b-8b84-85d8b7343d52" (UID: "7b24dda0-4030-406b-8b84-85d8b7343d52"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:28:19.041004 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.040981    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b24dda0-4030-406b-8b84-85d8b7343d52-kube-api-access-vrkbl" (OuterVolumeSpecName: "kube-api-access-vrkbl") pod "7b24dda0-4030-406b-8b84-85d8b7343d52" (UID: "7b24dda0-4030-406b-8b84-85d8b7343d52"). InnerVolumeSpecName "kube-api-access-vrkbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:28:19.041065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.041048    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d89146e6-db95-42e7-b963-5b39f3d1f60b-kube-api-access-hzz4g" (OuterVolumeSpecName: "kube-api-access-hzz4g") pod "d89146e6-db95-42e7-b963-5b39f3d1f60b" (UID: "d89146e6-db95-42e7-b963-5b39f3d1f60b"). InnerVolumeSpecName "kube-api-access-hzz4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:28:19.140179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.140139    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d89146e6-db95-42e7-b963-5b39f3d1f60b-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:28:19.140179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.140172    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrkbl\" (UniqueName: \"kubernetes.io/projected/7b24dda0-4030-406b-8b84-85d8b7343d52-kube-api-access-vrkbl\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:28:19.140179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.140181    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b24dda0-4030-406b-8b84-85d8b7343d52-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:28:19.140179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.140189    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzz4g\" (UniqueName: \"kubernetes.io/projected/d89146e6-db95-42e7-b963-5b39f3d1f60b-kube-api-access-hzz4g\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:28:19.787098 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.787062    2565 generic.go:356] "Generic (PLEG): container finished" podID="d89146e6-db95-42e7-b963-5b39f3d1f60b" containerID="f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de" exitCode=0
Jul 26 18:28:19.787595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.787106    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"
Jul 26 18:28:19.787595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.787150    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c" event={"ID":"d89146e6-db95-42e7-b963-5b39f3d1f60b","Type":"ContainerDied","Data":"f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de"}
Jul 26 18:28:19.787595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.787214    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c" event={"ID":"d89146e6-db95-42e7-b963-5b39f3d1f60b","Type":"ContainerDied","Data":"4d3338f3515caeec35979a914513912e6aaa1eaf624f137fee243ceb6a3c1d58"}
Jul 26 18:28:19.787595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.787251    2565 scope.go:117] "RemoveContainer" containerID="f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de"
Jul 26 18:28:19.788612 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.788508    2565 generic.go:356] "Generic (PLEG): container finished" podID="7b24dda0-4030-406b-8b84-85d8b7343d52" containerID="782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af" exitCode=0
Jul 26 18:28:19.788612 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.788557    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb" event={"ID":"7b24dda0-4030-406b-8b84-85d8b7343d52","Type":"ContainerDied","Data":"782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af"}
Jul 26 18:28:19.788612 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.788587    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb" event={"ID":"7b24dda0-4030-406b-8b84-85d8b7343d52","Type":"ContainerDied","Data":"90ffad34950c59aa72c015aebaa3631fe6fe3fb6629202ecd8fab37e8d977ab2"}
Jul 26 18:28:19.788612 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.788558    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"
Jul 26 18:28:19.796290 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.796184    2565 scope.go:117] "RemoveContainer" containerID="f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de"
Jul 26 18:28:19.796538 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:28:19.796517    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de\": container with ID starting with f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de not found: ID does not exist" containerID="f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de"
Jul 26 18:28:19.796634 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.796548    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de"} err="failed to get container status \"f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de\": rpc error: code = NotFound desc = could not find container \"f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de\": container with ID starting with f5aa445400849deac1484a28ecf41fcfd8db158572f1054a5a2ec71781b4a8de not found: ID does not exist"
Jul 26 18:28:19.796634 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.796570    2565 scope.go:117] "RemoveContainer" containerID="782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af"
Jul 26 18:28:19.804437 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.804423    2565 scope.go:117] "RemoveContainer" containerID="782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af"
Jul 26 18:28:19.804681 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:28:19.804665    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af\": container with ID starting with 782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af not found: ID does not exist" containerID="782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af"
Jul 26 18:28:19.804748 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.804687    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af"} err="failed to get container status \"782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af\": rpc error: code = NotFound desc = could not find container \"782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af\": container with ID starting with 782030c57fece9fa8b20608cc06ba2b258279304d8a373d3b835d5590ffdf3af not found: ID does not exist"
Jul 26 18:28:19.811691 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.811664    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"]
Jul 26 18:28:19.815977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.815959    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-ded9d893-8bb894d6b-wl49c"]
Jul 26 18:28:19.825713 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.825684    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"]
Jul 26 18:28:19.833127 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:19.833108    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ded9d893-74bdf55bd-f7nzb"]
Jul 26 18:28:20.318358 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:20.318327    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b24dda0-4030-406b-8b84-85d8b7343d52" path="/var/lib/kubelet/pods/7b24dda0-4030-406b-8b84-85d8b7343d52/volumes"
Jul 26 18:28:20.318663 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:20.318651    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d89146e6-db95-42e7-b963-5b39f3d1f60b" path="/var/lib/kubelet/pods/d89146e6-db95-42e7-b963-5b39f3d1f60b/volumes"
Jul 26 18:28:21.618199 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:21.618165    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:28:23.301795 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:23.301755    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"]
Jul 26 18:28:23.305259 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:23.305232    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-ded9d893-srj7k"]
Jul 26 18:28:24.318524 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:28:24.318497    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59339196-1dfd-4487-b894-22aa26c6ec38" path="/var/lib/kubelet/pods/59339196-1dfd-4487-b894-22aa26c6ec38/volumes"
Jul 26 18:29:49.216242 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:29:49.216203    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:29:58.520114 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:29:58.520080    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:30:00.169068 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169029    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"]
Jul 26 18:30:00.169549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169441    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b24dda0-4030-406b-8b84-85d8b7343d52" containerName="payload-pre-processing"
Jul 26 18:30:00.169549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169460    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b24dda0-4030-406b-8b84-85d8b7343d52" containerName="payload-pre-processing"
Jul 26 18:30:00.169549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169473    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59339196-1dfd-4487-b894-22aa26c6ec38" containerName="revoke-keys"
Jul 26 18:30:00.169549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169481    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="59339196-1dfd-4487-b894-22aa26c6ec38" containerName="revoke-keys"
Jul 26 18:30:00.169549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169498    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d89146e6-db95-42e7-b963-5b39f3d1f60b" containerName="payload-processing"
Jul 26 18:30:00.169549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169506    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d89146e6-db95-42e7-b963-5b39f3d1f60b" containerName="payload-processing"
Jul 26 18:30:00.169884 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169594    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="59339196-1dfd-4487-b894-22aa26c6ec38" containerName="revoke-keys"
Jul 26 18:30:00.169884 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169608    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b24dda0-4030-406b-8b84-85d8b7343d52" containerName="payload-pre-processing"
Jul 26 18:30:00.169884 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.169617    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d89146e6-db95-42e7-b963-5b39f3d1f60b" containerName="payload-processing"
Jul 26 18:30:00.172526 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.172503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"
Jul 26 18:30:00.177106 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.177085    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:30:00.181521 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.181503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"
Jul 26 18:30:00.206298 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.206269    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"]
Jul 26 18:30:00.334316 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:00.334292    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"]
Jul 26 18:30:00.336796 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:30:00.336769    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0197cc_dee3_4a9d_b4bb_b28b9c0e5bca.slice/crio-f830916cd8b7cc4bf6a4e31c4f6f23d56a02343bfb431c3e72c06ecc0a97dca4 WatchSource:0}: Error finding container f830916cd8b7cc4bf6a4e31c4f6f23d56a02343bfb431c3e72c06ecc0a97dca4: Status 404 returned error can't find the container with id f830916cd8b7cc4bf6a4e31c4f6f23d56a02343bfb431c3e72c06ecc0a97dca4
Jul 26 18:30:01.286549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:01.286520    2565 generic.go:356] "Generic (PLEG): container finished" podID="dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca" containerID="03b84c2e875d3e0bdacf529c5d068accfac808c8c187146d1fe162d380546eaf" exitCode=0
Jul 26 18:30:01.286938 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:01.286583    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6" event={"ID":"dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca","Type":"ContainerDied","Data":"03b84c2e875d3e0bdacf529c5d068accfac808c8c187146d1fe162d380546eaf"}
Jul 26 18:30:01.286938 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:01.286620    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6" event={"ID":"dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca","Type":"ContainerStarted","Data":"f830916cd8b7cc4bf6a4e31c4f6f23d56a02343bfb431c3e72c06ecc0a97dca4"}
Jul 26 18:30:02.420313 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:02.420278    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"
Jul 26 18:30:03.298138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:03.298111    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6"
Jul 26 18:30:03.298305 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:03.298140    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751510-g2dj6" event={"ID":"dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca","Type":"ContainerDied","Data":"f830916cd8b7cc4bf6a4e31c4f6f23d56a02343bfb431c3e72c06ecc0a97dca4"}
Jul 26 18:30:03.298305 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:03.298189    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f830916cd8b7cc4bf6a4e31c4f6f23d56a02343bfb431c3e72c06ecc0a97dca4"
Jul 26 18:30:04.925015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.924979    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"]
Jul 26 18:30:04.925400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.925275    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca" containerName="cleanup"
Jul 26 18:30:04.925400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.925285    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca" containerName="cleanup"
Jul 26 18:30:04.925400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.925340    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca" containerName="cleanup"
Jul 26 18:30:04.928154 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.928138    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:04.931009 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.930989    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2ef7c258\""
Jul 26 18:30:04.931165 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.931150    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2ef7c258-dockercfg-dwlq9\""
Jul 26 18:30:04.938987 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.938963    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"]
Jul 26 18:30:04.945546 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.945526    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"]
Jul 26 18:30:04.948640 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.948624    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:04.958966 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:04.958948    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"]
Jul 26 18:30:05.083239 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.083208    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.083416 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.083260    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hch4m\" (UniqueName: \"kubernetes.io/projected/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-kube-api-access-hch4m\") pod \"payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.083416 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.083350    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-plugins-config-volume\") pod \"payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") " pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.083416 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.083409    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7cqg\" (UniqueName: \"kubernetes.io/projected/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-kube-api-access-q7cqg\") pod \"payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") " pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.184750 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.184671    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7cqg\" (UniqueName: \"kubernetes.io/projected/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-kube-api-access-q7cqg\") pod \"payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") " pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.184879 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.184834    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.184922 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.184886    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hch4m\" (UniqueName: \"kubernetes.io/projected/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-kube-api-access-hch4m\") pod \"payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.184922 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.184919    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-plugins-config-volume\") pod \"payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") " pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.185513 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.185496    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.185549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.185502    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-plugins-config-volume\") pod \"payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") " pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.207346 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.207315    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hch4m\" (UniqueName: \"kubernetes.io/projected/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-kube-api-access-hch4m\") pod \"payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.207464 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.207409    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7cqg\" (UniqueName: \"kubernetes.io/projected/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-kube-api-access-q7cqg\") pod \"payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") " pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.237745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.237717    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:05.258465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.258433    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:05.381317 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.381288    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"]
Jul 26 18:30:05.384618 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:30:05.384587    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ab14c5_4b7b_47a6_afb5_f9582b2e7089.slice/crio-5548018d26122ddb1a19d4a4e8112c014342ab5b102d4792f06f3023d4833b8a WatchSource:0}: Error finding container 5548018d26122ddb1a19d4a4e8112c014342ab5b102d4792f06f3023d4833b8a: Status 404 returned error can't find the container with id 5548018d26122ddb1a19d4a4e8112c014342ab5b102d4792f06f3023d4833b8a
Jul 26 18:30:05.411666 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:05.411643    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"]
Jul 26 18:30:05.416831 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:30:05.416806    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38eeb9c6_c1e9_4bd1_8a44_bfbd34abb5f1.slice/crio-4d5e6282bfe6722b9a1702ac973352383ea8fa12b1bba55fbf1280ca1ba4b5da WatchSource:0}: Error finding container 4d5e6282bfe6722b9a1702ac973352383ea8fa12b1bba55fbf1280ca1ba4b5da: Status 404 returned error can't find the container with id 4d5e6282bfe6722b9a1702ac973352383ea8fa12b1bba55fbf1280ca1ba4b5da
Jul 26 18:30:06.319236 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.319194    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr" event={"ID":"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1","Type":"ContainerStarted","Data":"c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc"}
Jul 26 18:30:06.319639 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.319257    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:06.319639 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.319274    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr" event={"ID":"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1","Type":"ContainerStarted","Data":"4d5e6282bfe6722b9a1702ac973352383ea8fa12b1bba55fbf1280ca1ba4b5da"}
Jul 26 18:30:06.319639 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.319493    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4" event={"ID":"52ab14c5-4b7b-47a6-afb5-f9582b2e7089","Type":"ContainerStarted","Data":"3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c"}
Jul 26 18:30:06.319639 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.319522    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4" event={"ID":"52ab14c5-4b7b-47a6-afb5-f9582b2e7089","Type":"ContainerStarted","Data":"5548018d26122ddb1a19d4a4e8112c014342ab5b102d4792f06f3023d4833b8a"}
Jul 26 18:30:06.319639 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.319603    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:06.340498 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.340451    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr" podStartSLOduration=2.340437143 podStartE2EDuration="2.340437143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:06.338135077 +0000 UTC m=+1960.528164828" watchObservedRunningTime="2026-07-26 18:30:06.340437143 +0000 UTC m=+1960.530466896"
Jul 26 18:30:06.360832 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:06.360783    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4" podStartSLOduration=2.360769907 podStartE2EDuration="2.360769907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:06.358578978 +0000 UTC m=+1960.548608732" watchObservedRunningTime="2026-07-26 18:30:06.360769907 +0000 UTC m=+1960.550799689"
Jul 26 18:30:10.359157 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:10.359119    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:30:15.339360 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.339320    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"]
Jul 26 18:30:15.342751 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.342735    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:15.345644 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.345621    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:30:15.345774 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.345646    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:30:15.351575 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.351554    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"]
Jul 26 18:30:15.366427 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.366398    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br\" (UID: \"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:15.467042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.467010    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br\" (UID: \"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:15.467673 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.467651    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br\" (UID: \"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:15.654201 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.654107    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:15.786076 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:15.786051    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"]
Jul 26 18:30:15.787605 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:30:15.787581    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f665da_1da3_44d5_b3f9_4a8aa9d87cd8.slice/crio-5b46f9f0f110581c4a6bb12cdadef33576d4b045b33af4c0742973dd6a1e0db9 WatchSource:0}: Error finding container 5b46f9f0f110581c4a6bb12cdadef33576d4b045b33af4c0742973dd6a1e0db9: Status 404 returned error can't find the container with id 5b46f9f0f110581c4a6bb12cdadef33576d4b045b33af4c0742973dd6a1e0db9
Jul 26 18:30:16.369084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:16.368999    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br" event={"ID":"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8","Type":"ContainerStarted","Data":"51958bb54dbf90d2fa5c1a2242047beb730f91f0936d4a3fd6186c47bccb34f3"}
Jul 26 18:30:16.369084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:16.369067    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br" event={"ID":"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8","Type":"ContainerStarted","Data":"5b46f9f0f110581c4a6bb12cdadef33576d4b045b33af4c0742973dd6a1e0db9"}
Jul 26 18:30:16.386690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:16.386638    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br" podStartSLOduration=1.386621882 podStartE2EDuration="1.386621882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:30:16.385079293 +0000 UTC m=+1970.575109047" watchObservedRunningTime="2026-07-26 18:30:16.386621882 +0000 UTC m=+1970.576651635"
Jul 26 18:30:17.326908 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:17.326875    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:17.327108 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:17.326946    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:17.375598 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:17.375559    2565 generic.go:356] "Generic (PLEG): container finished" podID="98f665da-1da3-44d5-b3f9-4a8aa9d87cd8" containerID="51958bb54dbf90d2fa5c1a2242047beb730f91f0936d4a3fd6186c47bccb34f3" exitCode=0
Jul 26 18:30:17.376080 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:17.375624    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br" event={"ID":"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8","Type":"ContainerDied","Data":"51958bb54dbf90d2fa5c1a2242047beb730f91f0936d4a3fd6186c47bccb34f3"}
Jul 26 18:30:18.506316 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:18.506292    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:18.589486 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:18.589447    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8-maas-api-service-ca\") pod \"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8\" (UID: \"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8\") "
Jul 26 18:30:18.589807 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:18.589785    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "98f665da-1da3-44d5-b3f9-4a8aa9d87cd8" (UID: "98f665da-1da3-44d5-b3f9-4a8aa9d87cd8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:30:18.690756 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:18.690649    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:30:19.387017 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:19.386987    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"
Jul 26 18:30:19.387257 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:19.386990    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br" event={"ID":"98f665da-1da3-44d5-b3f9-4a8aa9d87cd8","Type":"ContainerDied","Data":"5b46f9f0f110581c4a6bb12cdadef33576d4b045b33af4c0742973dd6a1e0db9"}
Jul 26 18:30:19.387257 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:19.387102    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b46f9f0f110581c4a6bb12cdadef33576d4b045b33af4c0742973dd6a1e0db9"
Jul 26 18:30:25.536651 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.536613    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"]
Jul 26 18:30:25.537183 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.536836    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr" podUID="38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" containerName="payload-processing" containerID="cri-o://c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc" gracePeriod=30
Jul 26 18:30:25.542505 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.542472    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"]
Jul 26 18:30:25.542745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.542716    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4" podUID="52ab14c5-4b7b-47a6-afb5-f9582b2e7089" containerName="payload-pre-processing" containerID="cri-o://3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c" gracePeriod=30
Jul 26 18:30:25.811734 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.811710    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:25.814211 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.814190    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:25.841241 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.841210    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hch4m\" (UniqueName: \"kubernetes.io/projected/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-kube-api-access-hch4m\") pod \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") "
Jul 26 18:30:25.841433 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.841249    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7cqg\" (UniqueName: \"kubernetes.io/projected/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-kube-api-access-q7cqg\") pod \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") "
Jul 26 18:30:25.841433 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.841293    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-plugins-config-volume\") pod \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\" (UID: \"52ab14c5-4b7b-47a6-afb5-f9582b2e7089\") "
Jul 26 18:30:25.841433 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.841338    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-plugins-config-volume\") pod \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\" (UID: \"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1\") "
Jul 26 18:30:25.841769 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.841736    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "52ab14c5-4b7b-47a6-afb5-f9582b2e7089" (UID: "52ab14c5-4b7b-47a6-afb5-f9582b2e7089"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:30:25.841769 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.841755    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" (UID: "38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:30:25.843328 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.843295    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-kube-api-access-hch4m" (OuterVolumeSpecName: "kube-api-access-hch4m") pod "52ab14c5-4b7b-47a6-afb5-f9582b2e7089" (UID: "52ab14c5-4b7b-47a6-afb5-f9582b2e7089"). InnerVolumeSpecName "kube-api-access-hch4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:30:25.843443 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.843391    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-kube-api-access-q7cqg" (OuterVolumeSpecName: "kube-api-access-q7cqg") pod "38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" (UID: "38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1"). InnerVolumeSpecName "kube-api-access-q7cqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:30:25.942847 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.942813    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hch4m\" (UniqueName: \"kubernetes.io/projected/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-kube-api-access-hch4m\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:30:25.942847 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.942840    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7cqg\" (UniqueName: \"kubernetes.io/projected/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-kube-api-access-q7cqg\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:30:25.943050 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.942849    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/52ab14c5-4b7b-47a6-afb5-f9582b2e7089-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:30:25.943050 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:25.942876    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:30:26.423022 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.422985    2565 generic.go:356] "Generic (PLEG): container finished" podID="52ab14c5-4b7b-47a6-afb5-f9582b2e7089" containerID="3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c" exitCode=0
Jul 26 18:30:26.423197 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.423039    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"
Jul 26 18:30:26.423197 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.423078    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4" event={"ID":"52ab14c5-4b7b-47a6-afb5-f9582b2e7089","Type":"ContainerDied","Data":"3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c"}
Jul 26 18:30:26.423197 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.423139    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4" event={"ID":"52ab14c5-4b7b-47a6-afb5-f9582b2e7089","Type":"ContainerDied","Data":"5548018d26122ddb1a19d4a4e8112c014342ab5b102d4792f06f3023d4833b8a"}
Jul 26 18:30:26.423197 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.423162    2565 scope.go:117] "RemoveContainer" containerID="3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c"
Jul 26 18:30:26.424625 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.424551    2565 generic.go:356] "Generic (PLEG): container finished" podID="38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" containerID="c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc" exitCode=0
Jul 26 18:30:26.424625 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.424610    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"
Jul 26 18:30:26.424625 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.424621    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr" event={"ID":"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1","Type":"ContainerDied","Data":"c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc"}
Jul 26 18:30:26.424880 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.424655    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr" event={"ID":"38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1","Type":"ContainerDied","Data":"4d5e6282bfe6722b9a1702ac973352383ea8fa12b1bba55fbf1280ca1ba4b5da"}
Jul 26 18:30:26.431538 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.431519    2565 scope.go:117] "RemoveContainer" containerID="3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c"
Jul 26 18:30:26.431832 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:30:26.431813    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c\": container with ID starting with 3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c not found: ID does not exist" containerID="3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c"
Jul 26 18:30:26.431902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.431846    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c"} err="failed to get container status \"3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c\": rpc error: code = NotFound desc = could not find container \"3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c\": container with ID starting with 3b55faf40dd7b07759512e7965367cfeaba54094ffc33efd511748937b5d351c not found: ID does not exist"
Jul 26 18:30:26.431902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.431864    2565 scope.go:117] "RemoveContainer" containerID="c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc"
Jul 26 18:30:26.439446 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.439406    2565 scope.go:117] "RemoveContainer" containerID="c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc"
Jul 26 18:30:26.439709 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:30:26.439680    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc\": container with ID starting with c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc not found: ID does not exist" containerID="c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc"
Jul 26 18:30:26.439768 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.439732    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc"} err="failed to get container status \"c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc\": rpc error: code = NotFound desc = could not find container \"c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc\": container with ID starting with c20f090bd135c0acb455a4257ea2e0147fc444d29e873ffba1a8f57226a0c9fc not found: ID does not exist"
Jul 26 18:30:26.454056 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.454034    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"]
Jul 26 18:30:26.460625 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.460586    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2ef7c258-7d899f9b8d-6xht4"]
Jul 26 18:30:26.471672 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.471644    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"]
Jul 26 18:30:26.476398 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:26.476377    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2ef7c258-7887db9894-szgpr"]
Jul 26 18:30:28.318093 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:28.318067    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" path="/var/lib/kubelet/pods/38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1/volumes"
Jul 26 18:30:28.318464 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:28.318429    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52ab14c5-4b7b-47a6-afb5-f9582b2e7089" path="/var/lib/kubelet/pods/52ab14c5-4b7b-47a6-afb5-f9582b2e7089/volumes"
Jul 26 18:30:28.517015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:28.516974    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:30:30.428517 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:30.428469    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"]
Jul 26 18:30:30.431599 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:30.431570    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2ef7c258-zf6br"]
Jul 26 18:30:32.325022 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:30:32.324989    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98f665da-1da3-44d5-b3f9-4a8aa9d87cd8" path="/var/lib/kubelet/pods/98f665da-1da3-44d5-b3f9-4a8aa9d87cd8/volumes"
Jul 26 18:31:00.858772 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.858737    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"]
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859032    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" containerName="payload-processing"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859042    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" containerName="payload-processing"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859051    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98f665da-1da3-44d5-b3f9-4a8aa9d87cd8" containerName="revoke-keys"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859057    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="98f665da-1da3-44d5-b3f9-4a8aa9d87cd8" containerName="revoke-keys"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859068    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52ab14c5-4b7b-47a6-afb5-f9582b2e7089" containerName="payload-pre-processing"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859074    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="52ab14c5-4b7b-47a6-afb5-f9582b2e7089" containerName="payload-pre-processing"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859121    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="52ab14c5-4b7b-47a6-afb5-f9582b2e7089" containerName="payload-pre-processing"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859128    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="38eeb9c6-c1e9-4bd1-8a44-bfbd34abb5f1" containerName="payload-processing"
Jul 26 18:31:00.859250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.859133    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="98f665da-1da3-44d5-b3f9-4a8aa9d87cd8" containerName="revoke-keys"
Jul 26 18:31:00.861943 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.861921    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:00.864259 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.864239    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3a0f87c2\""
Jul 26 18:31:00.864504 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.864488    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3a0f87c2-dockercfg-q9jzg\""
Jul 26 18:31:00.873956 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.873931    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"]
Jul 26 18:31:00.883534 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.883513    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"]
Jul 26 18:31:00.886616 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.886600    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:00.896427 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.896408    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"]
Jul 26 18:31:00.935494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.935445    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v2cg\" (UniqueName: \"kubernetes.io/projected/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-kube-api-access-7v2cg\") pod \"payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") " pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:00.935682 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:00.935601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-plugins-config-volume\") pod \"payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") " pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.037141 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.037100    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v2cg\" (UniqueName: \"kubernetes.io/projected/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-kube-api-access-7v2cg\") pod \"payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") " pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.037474 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.037455    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-plugins-config-volume\") pod \"payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") " pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.037660 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.037633    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72df6ffc-5de0-4411-af20-4d52ff4d205f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.037810 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.037795    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkp42\" (UniqueName: \"kubernetes.io/projected/72df6ffc-5de0-4411-af20-4d52ff4d205f-kube-api-access-vkp42\") pod \"payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.039513 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.039478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-plugins-config-volume\") pod \"payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") " pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.047502 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.047467    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v2cg\" (UniqueName: \"kubernetes.io/projected/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-kube-api-access-7v2cg\") pod \"payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") " pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.138547 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.138451    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72df6ffc-5de0-4411-af20-4d52ff4d205f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.138547 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.138505    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkp42\" (UniqueName: \"kubernetes.io/projected/72df6ffc-5de0-4411-af20-4d52ff4d205f-kube-api-access-vkp42\") pod \"payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.139263 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.139242    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72df6ffc-5de0-4411-af20-4d52ff4d205f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.150154 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.150133    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkp42\" (UniqueName: \"kubernetes.io/projected/72df6ffc-5de0-4411-af20-4d52ff4d205f-kube-api-access-vkp42\") pod \"payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.172585 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.172552    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.197202 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.197166    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.326451 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.326423    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"]
Jul 26 18:31:01.328176 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:31:01.328140    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fc7eb98_72b7_4189_9c1d_556e97cdb6b3.slice/crio-0e4c592c4e1f736619a044a229e567313e1c65ef140e2502c7c8818793f92473 WatchSource:0}: Error finding container 0e4c592c4e1f736619a044a229e567313e1c65ef140e2502c7c8818793f92473: Status 404 returned error can't find the container with id 0e4c592c4e1f736619a044a229e567313e1c65ef140e2502c7c8818793f92473
Jul 26 18:31:01.346872 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.346849    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"]
Jul 26 18:31:01.348737 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:31:01.348691    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72df6ffc_5de0_4411_af20_4d52ff4d205f.slice/crio-84c21e0cf1cc3dfae479431cee5d9f403791f41469007ee3a2f3664013d83cef WatchSource:0}: Error finding container 84c21e0cf1cc3dfae479431cee5d9f403791f41469007ee3a2f3664013d83cef: Status 404 returned error can't find the container with id 84c21e0cf1cc3dfae479431cee5d9f403791f41469007ee3a2f3664013d83cef
Jul 26 18:31:01.598504 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.598458    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf" event={"ID":"72df6ffc-5de0-4411-af20-4d52ff4d205f","Type":"ContainerStarted","Data":"64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4"}
Jul 26 18:31:01.598687 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.598522    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf" event={"ID":"72df6ffc-5de0-4411-af20-4d52ff4d205f","Type":"ContainerStarted","Data":"84c21e0cf1cc3dfae479431cee5d9f403791f41469007ee3a2f3664013d83cef"}
Jul 26 18:31:01.598687 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.598643    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:01.600055 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.600027    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz" event={"ID":"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3","Type":"ContainerStarted","Data":"2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc"}
Jul 26 18:31:01.600161 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.600073    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz" event={"ID":"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3","Type":"ContainerStarted","Data":"0e4c592c4e1f736619a044a229e567313e1c65ef140e2502c7c8818793f92473"}
Jul 26 18:31:01.600161 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.600135    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:01.614179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.614128    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf" podStartSLOduration=1.614083 podStartE2EDuration="1.614083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:31:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:31:01.613235866 +0000 UTC m=+2015.803265620" watchObservedRunningTime="2026-07-26 18:31:01.614083 +0000 UTC m=+2015.804112754"
Jul 26 18:31:01.629910 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:01.629843    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz" podStartSLOduration=1.629826585 podStartE2EDuration="1.629826585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:31:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:31:01.62775602 +0000 UTC m=+2015.817785775" watchObservedRunningTime="2026-07-26 18:31:01.629826585 +0000 UTC m=+2015.819856338"
Jul 26 18:31:05.518081 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:05.518043    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:31:12.606952 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:12.606915    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:31:12.607337 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:12.607009    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:31:19.632250 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.632213    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"]
Jul 26 18:31:19.635737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.635719    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.638959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.638938    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3a0f87c2\"/\"openshift-service-ca.crt\""
Jul 26 18:31:19.639053 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.638938    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3a0f87c2\"/\"default-dockercfg-8pdl2\""
Jul 26 18:31:19.639923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.639906    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3a0f87c2\"/\"kube-root-ca.crt\""
Jul 26 18:31:19.640247 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.640231    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3a0f87c2\"/\"e2e-lifecycle-model-3a0f87c2-kserve-self-signed-certs\""
Jul 26 18:31:19.651848 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.651826    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"]
Jul 26 18:31:19.803858 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.803821    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69517955-43ac-4ff1-ab43-a5c713efa505-tls-certs\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.804059 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.803864    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-model-cache\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.804059 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.803894    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-home\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.804059 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.803945    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-tmp-dir\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.804059 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.804000    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-dshm\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.804059 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.804032    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rmmf\" (UniqueName: \"kubernetes.io/projected/69517955-43ac-4ff1-ab43-a5c713efa505-kube-api-access-8rmmf\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.904514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.904417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-dshm\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.904514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.904460    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rmmf\" (UniqueName: \"kubernetes.io/projected/69517955-43ac-4ff1-ab43-a5c713efa505-kube-api-access-8rmmf\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.904790 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.904535    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69517955-43ac-4ff1-ab43-a5c713efa505-tls-certs\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.904790 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.904567    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-model-cache\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.904790 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.904605    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-home\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.904790 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.904625    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-tmp-dir\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.905038 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.905015    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-home\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.905099 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.905054    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-model-cache\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.905099 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.905071    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-tmp-dir\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.906885 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.906861    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-dshm\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.907086 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.907069    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69517955-43ac-4ff1-ab43-a5c713efa505-tls-certs\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.913126 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.913101    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rmmf\" (UniqueName: \"kubernetes.io/projected/69517955-43ac-4ff1-ab43-a5c713efa505-kube-api-access-8rmmf\") pod \"e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") " pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:19.944924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:19.944886    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:20.076843 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:20.076811    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"]
Jul 26 18:31:20.079901 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:31:20.079878    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69517955_43ac_4ff1_ab43_a5c713efa505.slice/crio-0e13db3e7f5a11a053fc479db6c0c4aea1bb524e2b1a6ecf7aab92028f98bf4d WatchSource:0}: Error finding container 0e13db3e7f5a11a053fc479db6c0c4aea1bb524e2b1a6ecf7aab92028f98bf4d: Status 404 returned error can't find the container with id 0e13db3e7f5a11a053fc479db6c0c4aea1bb524e2b1a6ecf7aab92028f98bf4d
Jul 26 18:31:20.700674 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:20.700638    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw" event={"ID":"69517955-43ac-4ff1-ab43-a5c713efa505","Type":"ContainerStarted","Data":"565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435"}
Jul 26 18:31:20.701090 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:20.700683    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw" event={"ID":"69517955-43ac-4ff1-ab43-a5c713efa505","Type":"ContainerStarted","Data":"0e13db3e7f5a11a053fc479db6c0c4aea1bb524e2b1a6ecf7aab92028f98bf4d"}
Jul 26 18:31:20.717415 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:20.717368    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw" podStartSLOduration=1.7173538910000001 podStartE2EDuration="1.717353891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:31:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:31:20.716385166 +0000 UTC m=+2034.906414921" watchObservedRunningTime="2026-07-26 18:31:20.717353891 +0000 UTC m=+2034.907383645"
Jul 26 18:31:22.915662 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:22.915621    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:31:29.945300 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:29.945259    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:29.945722 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:29.945326    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:29.957748 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:29.957723    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:30.761949 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:30.761923    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:31:55.324674 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:55.324586    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:31:59.321427 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.321392    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"]
Jul 26 18:31:59.325331 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.325312    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:31:59.329383 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.329356    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:31:59.329510 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.329412    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:31:59.335773 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.335751    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"]
Jul 26 18:31:59.433844 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.433801    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv\" (UID: \"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:31:59.535189 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.535155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv\" (UID: \"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:31:59.535813 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.535795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv\" (UID: \"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:31:59.635676 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.635576    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:31:59.774080 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.774056    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"]
Jul 26 18:31:59.775682 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:31:59.775657    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60f1474b_c451_47cc_bb0a_5b9a5a13ce8d.slice/crio-0c6c895a52505b3124723102a845de1ba11e4bb5e89c669a7ab3b21c81e22128 WatchSource:0}: Error finding container 0c6c895a52505b3124723102a845de1ba11e4bb5e89c669a7ab3b21c81e22128: Status 404 returned error can't find the container with id 0c6c895a52505b3124723102a845de1ba11e4bb5e89c669a7ab3b21c81e22128
Jul 26 18:31:59.902591 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:31:59.902516    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv" event={"ID":"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d","Type":"ContainerStarted","Data":"0c6c895a52505b3124723102a845de1ba11e4bb5e89c669a7ab3b21c81e22128"}
Jul 26 18:32:00.909266 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:00.909229    2565 generic.go:356] "Generic (PLEG): container finished" podID="60f1474b-c451-47cc-bb0a-5b9a5a13ce8d" containerID="7a61aa86ed5b867ccf61bb637de2fc244f36577880a9d90e8b1d47c910cdc8a0" exitCode=0
Jul 26 18:32:00.909685 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:00.909318    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv" event={"ID":"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d","Type":"ContainerDied","Data":"7a61aa86ed5b867ccf61bb637de2fc244f36577880a9d90e8b1d47c910cdc8a0"}
Jul 26 18:32:02.043511 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.043488    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:32:02.157123 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.157086    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d-maas-api-service-ca\") pod \"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d\" (UID: \"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d\") "
Jul 26 18:32:02.157474 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.157450    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "60f1474b-c451-47cc-bb0a-5b9a5a13ce8d" (UID: "60f1474b-c451-47cc-bb0a-5b9a5a13ce8d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:32:02.257831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.257794    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:02.923062 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.923034    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"
Jul 26 18:32:02.923062 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.923038    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv" event={"ID":"60f1474b-c451-47cc-bb0a-5b9a5a13ce8d","Type":"ContainerDied","Data":"0c6c895a52505b3124723102a845de1ba11e4bb5e89c669a7ab3b21c81e22128"}
Jul 26 18:32:02.923267 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:02.923085    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c6c895a52505b3124723102a845de1ba11e4bb5e89c669a7ab3b21c81e22128"
Jul 26 18:32:13.427778 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:13.427741    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:32:14.890111 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:14.890076    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"]
Jul 26 18:32:14.890534 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:14.890319    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz" podUID="6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" containerName="payload-processing" containerID="cri-o://2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc" gracePeriod=30
Jul 26 18:32:14.894949 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:14.894921    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"]
Jul 26 18:32:14.895247 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:14.895205    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf" podUID="72df6ffc-5de0-4411-af20-4d52ff4d205f" containerName="payload-pre-processing" containerID="cri-o://64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4" gracePeriod=30
Jul 26 18:32:15.140681 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.140620    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:32:15.172376 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.172348    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:32:15.267055 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.267011    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72df6ffc-5de0-4411-af20-4d52ff4d205f-plugins-config-volume\") pod \"72df6ffc-5de0-4411-af20-4d52ff4d205f\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") "
Jul 26 18:32:15.267055 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.267073    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-plugins-config-volume\") pod \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") "
Jul 26 18:32:15.267343 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.267094    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkp42\" (UniqueName: \"kubernetes.io/projected/72df6ffc-5de0-4411-af20-4d52ff4d205f-kube-api-access-vkp42\") pod \"72df6ffc-5de0-4411-af20-4d52ff4d205f\" (UID: \"72df6ffc-5de0-4411-af20-4d52ff4d205f\") "
Jul 26 18:32:15.267343 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.267141    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v2cg\" (UniqueName: \"kubernetes.io/projected/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-kube-api-access-7v2cg\") pod \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\" (UID: \"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3\") "
Jul 26 18:32:15.267456 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.267435    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72df6ffc-5de0-4411-af20-4d52ff4d205f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "72df6ffc-5de0-4411-af20-4d52ff4d205f" (UID: "72df6ffc-5de0-4411-af20-4d52ff4d205f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:32:15.267506 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.267473    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" (UID: "6fc7eb98-72b7-4189-9c1d-556e97cdb6b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:32:15.269247 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.269214    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72df6ffc-5de0-4411-af20-4d52ff4d205f-kube-api-access-vkp42" (OuterVolumeSpecName: "kube-api-access-vkp42") pod "72df6ffc-5de0-4411-af20-4d52ff4d205f" (UID: "72df6ffc-5de0-4411-af20-4d52ff4d205f"). InnerVolumeSpecName "kube-api-access-vkp42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:32:15.269375 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.269272    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-kube-api-access-7v2cg" (OuterVolumeSpecName: "kube-api-access-7v2cg") pod "6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" (UID: "6fc7eb98-72b7-4189-9c1d-556e97cdb6b3"). InnerVolumeSpecName "kube-api-access-7v2cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:32:15.368794 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.368755    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72df6ffc-5de0-4411-af20-4d52ff4d205f-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:15.368794 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.368794    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:15.368988 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.368809    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkp42\" (UniqueName: \"kubernetes.io/projected/72df6ffc-5de0-4411-af20-4d52ff4d205f-kube-api-access-vkp42\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:15.368988 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.368825    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v2cg\" (UniqueName: \"kubernetes.io/projected/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3-kube-api-access-7v2cg\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:15.994447 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.994413    2565 generic.go:356] "Generic (PLEG): container finished" podID="6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" containerID="2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc" exitCode=0
Jul 26 18:32:15.994887 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.994468    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"
Jul 26 18:32:15.994887 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.994484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz" event={"ID":"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3","Type":"ContainerDied","Data":"2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc"}
Jul 26 18:32:15.994887 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.994533    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz" event={"ID":"6fc7eb98-72b7-4189-9c1d-556e97cdb6b3","Type":"ContainerDied","Data":"0e4c592c4e1f736619a044a229e567313e1c65ef140e2502c7c8818793f92473"}
Jul 26 18:32:15.994887 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.994555    2565 scope.go:117] "RemoveContainer" containerID="2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc"
Jul 26 18:32:15.995934 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.995840    2565 generic.go:356] "Generic (PLEG): container finished" podID="72df6ffc-5de0-4411-af20-4d52ff4d205f" containerID="64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4" exitCode=0
Jul 26 18:32:15.995934 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.995875    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf" event={"ID":"72df6ffc-5de0-4411-af20-4d52ff4d205f","Type":"ContainerDied","Data":"64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4"}
Jul 26 18:32:15.995934 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.995895    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"
Jul 26 18:32:15.995934 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:15.995928    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf" event={"ID":"72df6ffc-5de0-4411-af20-4d52ff4d205f","Type":"ContainerDied","Data":"84c21e0cf1cc3dfae479431cee5d9f403791f41469007ee3a2f3664013d83cef"}
Jul 26 18:32:16.009332 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.009312    2565 scope.go:117] "RemoveContainer" containerID="2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc"
Jul 26 18:32:16.009731 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:32:16.009687    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc\": container with ID starting with 2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc not found: ID does not exist" containerID="2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc"
Jul 26 18:32:16.009812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.009739    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc"} err="failed to get container status \"2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc\": rpc error: code = NotFound desc = could not find container \"2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc\": container with ID starting with 2766e406b4de17e9ba8119bc11864ecf6657284e28c1a1452b9cfcbd30ee2bdc not found: ID does not exist"
Jul 26 18:32:16.009812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.009757    2565 scope.go:117] "RemoveContainer" containerID="64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4"
Jul 26 18:32:16.017898 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.017879    2565 scope.go:117] "RemoveContainer" containerID="64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4"
Jul 26 18:32:16.018167 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:32:16.018150    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4\": container with ID starting with 64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4 not found: ID does not exist" containerID="64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4"
Jul 26 18:32:16.018213 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.018175    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4"} err="failed to get container status \"64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4\": rpc error: code = NotFound desc = could not find container \"64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4\": container with ID starting with 64b4daa766c82ba86cec19ddecc57751a4fc9eee64014b7a333468cf9a01e8f4 not found: ID does not exist"
Jul 26 18:32:16.024138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.024116    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"]
Jul 26 18:32:16.029671 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.029652    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-3a0f87c2-88d749766-5vnpz"]
Jul 26 18:32:16.039923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.039891    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"]
Jul 26 18:32:16.045366 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.045344    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3a0f87c2-7499554c96-m2qvf"]
Jul 26 18:32:16.319128 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.319089    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" path="/var/lib/kubelet/pods/6fc7eb98-72b7-4189-9c1d-556e97cdb6b3/volumes"
Jul 26 18:32:16.319539 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:16.319520    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72df6ffc-5de0-4411-af20-4d52ff4d205f" path="/var/lib/kubelet/pods/72df6ffc-5de0-4411-af20-4d52ff4d205f/volumes"
Jul 26 18:32:19.420665 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:19.420633    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"]
Jul 26 18:32:19.422616 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:19.422589    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0f87c2-tvnjv"]
Jul 26 18:32:20.319212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:20.319174    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60f1474b-c451-47cc-bb0a-5b9a5a13ce8d" path="/var/lib/kubelet/pods/60f1474b-c451-47cc-bb0a-5b9a5a13ce8d/volumes"
Jul 26 18:32:22.709189 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:22.709152    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:32:26.390107 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:26.390077    2565 scope.go:117] "RemoveContainer" containerID="0119645a1e914a4f169d6562e863b90f42d673afea2aec79554ab14f89bc0a7f"
Jul 26 18:32:27.497637 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:27.497593    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"]
Jul 26 18:32:42.516330 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.516282    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw" podUID="69517955-43ac-4ff1-ab43-a5c713efa505" containerName="main" containerID="cri-o://565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435" gracePeriod=45
Jul 26 18:32:42.767492 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.767430    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:32:42.880931 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.880900    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rmmf\" (UniqueName: \"kubernetes.io/projected/69517955-43ac-4ff1-ab43-a5c713efa505-kube-api-access-8rmmf\") pod \"69517955-43ac-4ff1-ab43-a5c713efa505\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") "
Jul 26 18:32:42.881138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.880954    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-model-cache\") pod \"69517955-43ac-4ff1-ab43-a5c713efa505\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") "
Jul 26 18:32:42.881138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881007    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-tmp-dir\") pod \"69517955-43ac-4ff1-ab43-a5c713efa505\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") "
Jul 26 18:32:42.881138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881096    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-dshm\") pod \"69517955-43ac-4ff1-ab43-a5c713efa505\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") "
Jul 26 18:32:42.881138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881129    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-home\") pod \"69517955-43ac-4ff1-ab43-a5c713efa505\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") "
Jul 26 18:32:42.881337 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881166    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69517955-43ac-4ff1-ab43-a5c713efa505-tls-certs\") pod \"69517955-43ac-4ff1-ab43-a5c713efa505\" (UID: \"69517955-43ac-4ff1-ab43-a5c713efa505\") "
Jul 26 18:32:42.881337 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881250    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-model-cache" (OuterVolumeSpecName: "model-cache") pod "69517955-43ac-4ff1-ab43-a5c713efa505" (UID: "69517955-43ac-4ff1-ab43-a5c713efa505"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:32:42.881337 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881264    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "69517955-43ac-4ff1-ab43-a5c713efa505" (UID: "69517955-43ac-4ff1-ab43-a5c713efa505"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:32:42.881450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881414    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:42.881496 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881475    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:42.881496 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.881412    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-home" (OuterVolumeSpecName: "home") pod "69517955-43ac-4ff1-ab43-a5c713efa505" (UID: "69517955-43ac-4ff1-ab43-a5c713efa505"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:32:42.883324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.883278    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-dshm" (OuterVolumeSpecName: "dshm") pod "69517955-43ac-4ff1-ab43-a5c713efa505" (UID: "69517955-43ac-4ff1-ab43-a5c713efa505"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:32:42.883324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.883291    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69517955-43ac-4ff1-ab43-a5c713efa505-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "69517955-43ac-4ff1-ab43-a5c713efa505" (UID: "69517955-43ac-4ff1-ab43-a5c713efa505"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:32:42.883324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.883309    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69517955-43ac-4ff1-ab43-a5c713efa505-kube-api-access-8rmmf" (OuterVolumeSpecName: "kube-api-access-8rmmf") pod "69517955-43ac-4ff1-ab43-a5c713efa505" (UID: "69517955-43ac-4ff1-ab43-a5c713efa505"). InnerVolumeSpecName "kube-api-access-8rmmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:32:42.982018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.981979    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:42.982018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.982007    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/69517955-43ac-4ff1-ab43-a5c713efa505-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:42.982018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.982016    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/69517955-43ac-4ff1-ab43-a5c713efa505-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:42.982018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:42.982027    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rmmf\" (UniqueName: \"kubernetes.io/projected/69517955-43ac-4ff1-ab43-a5c713efa505-kube-api-access-8rmmf\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:32:43.133060 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.132974    2565 generic.go:356] "Generic (PLEG): container finished" podID="69517955-43ac-4ff1-ab43-a5c713efa505" containerID="565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435" exitCode=0
Jul 26 18:32:43.133060 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.133016    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"
Jul 26 18:32:43.133251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.133059    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw" event={"ID":"69517955-43ac-4ff1-ab43-a5c713efa505","Type":"ContainerDied","Data":"565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435"}
Jul 26 18:32:43.133251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.133106    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw" event={"ID":"69517955-43ac-4ff1-ab43-a5c713efa505","Type":"ContainerDied","Data":"0e13db3e7f5a11a053fc479db6c0c4aea1bb524e2b1a6ecf7aab92028f98bf4d"}
Jul 26 18:32:43.133251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.133128    2565 scope.go:117] "RemoveContainer" containerID="565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435"
Jul 26 18:32:43.141718 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.141682    2565 scope.go:117] "RemoveContainer" containerID="565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435"
Jul 26 18:32:43.142044 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:32:43.142024    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435\": container with ID starting with 565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435 not found: ID does not exist" containerID="565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435"
Jul 26 18:32:43.142171 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.142055    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435"} err="failed to get container status \"565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435\": rpc error: code = NotFound desc = could not find container \"565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435\": container with ID starting with 565213210b093e8fe2652ad8bf51a538fd67a270bfa4b4da7f656c0d3ddb7435 not found: ID does not exist"
Jul 26 18:32:43.154946 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.154911    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"]
Jul 26 18:32:43.158545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:43.158518    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3a0f87c2/e2e-lifecycle-model-3a0f87c2-kserve-6f6556fdcf-8rltw"]
Jul 26 18:32:44.318881 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:32:44.318850    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69517955-43ac-4ff1-ab43-a5c713efa505" path="/var/lib/kubelet/pods/69517955-43ac-4ff1-ab43-a5c713efa505/volumes"
Jul 26 18:33:20.538164 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:33:20.538083    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:33:26.416485 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:33:26.416449    2565 scope.go:117] "RemoveContainer" containerID="86be99aacf9934e563898dd3998e1a26c04848f3ff97ecf344cc18f7d0eb401c"
Jul 26 18:33:26.424633 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:33:26.424613    2565 scope.go:117] "RemoveContainer" containerID="a5b187ff999d20dfadf5166a8e1f4d6c9cc2c42b6651a3bb1f9324363d3faff5"
Jul 26 18:33:32.347645 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:33:32.347604    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:34:16.741715 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.741665    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"]
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.741961    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69517955-43ac-4ff1-ab43-a5c713efa505" containerName="main"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.741972    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="69517955-43ac-4ff1-ab43-a5c713efa505" containerName="main"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.741981    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" containerName="payload-processing"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.741987    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" containerName="payload-processing"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742001    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60f1474b-c451-47cc-bb0a-5b9a5a13ce8d" containerName="revoke-keys"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742008    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="60f1474b-c451-47cc-bb0a-5b9a5a13ce8d" containerName="revoke-keys"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742015    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72df6ffc-5de0-4411-af20-4d52ff4d205f" containerName="payload-pre-processing"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742021    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="72df6ffc-5de0-4411-af20-4d52ff4d205f" containerName="payload-pre-processing"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742068    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="60f1474b-c451-47cc-bb0a-5b9a5a13ce8d" containerName="revoke-keys"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742077    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="72df6ffc-5de0-4411-af20-4d52ff4d205f" containerName="payload-pre-processing"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742082    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fc7eb98-72b7-4189-9c1d-556e97cdb6b3" containerName="payload-processing"
Jul 26 18:34:16.742146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.742087    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="69517955-43ac-4ff1-ab43-a5c713efa505" containerName="main"
Jul 26 18:34:16.744929 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.744910    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:16.747844 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.747824    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-872031\""
Jul 26 18:34:16.756324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.756296    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"]
Jul 26 18:34:16.789213 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.789186    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"]
Jul 26 18:34:16.792522 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.792503    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:16.795149 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.795125    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-872031-dockercfg-vcv6v\""
Jul 26 18:34:16.804606 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.804578    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"]
Jul 26 18:34:16.824800 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.824765    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb82aa3e-e46e-4f8e-914e-fcfc67128310-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:16.824963 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.824815    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnnwr\" (UniqueName: \"kubernetes.io/projected/eb82aa3e-e46e-4f8e-914e-fcfc67128310-kube-api-access-rnnwr\") pod \"payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:16.926088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.926045    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b26963-d067-40e3-b6f3-0ab5e88da2fa-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:16.926300 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.926120    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb82aa3e-e46e-4f8e-914e-fcfc67128310-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:16.926300 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.926148    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlrzq\" (UniqueName: \"kubernetes.io/projected/82b26963-d067-40e3-b6f3-0ab5e88da2fa-kube-api-access-jlrzq\") pod \"payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:16.926300 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.926188    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnnwr\" (UniqueName: \"kubernetes.io/projected/eb82aa3e-e46e-4f8e-914e-fcfc67128310-kube-api-access-rnnwr\") pod \"payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:16.926913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.926892    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb82aa3e-e46e-4f8e-914e-fcfc67128310-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:16.934760 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:16.934736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnnwr\" (UniqueName: \"kubernetes.io/projected/eb82aa3e-e46e-4f8e-914e-fcfc67128310-kube-api-access-rnnwr\") pod \"payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:17.027117 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.027022    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b26963-d067-40e3-b6f3-0ab5e88da2fa-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:17.027117 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.027085    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlrzq\" (UniqueName: \"kubernetes.io/projected/82b26963-d067-40e3-b6f3-0ab5e88da2fa-kube-api-access-jlrzq\") pod \"payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:17.027777 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.027753    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b26963-d067-40e3-b6f3-0ab5e88da2fa-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:17.038863 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.038838    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlrzq\" (UniqueName: \"kubernetes.io/projected/82b26963-d067-40e3-b6f3-0ab5e88da2fa-kube-api-access-jlrzq\") pod \"payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") " pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:17.054712 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.054682    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:17.102153 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.102121    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:17.198227 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.198131    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"]
Jul 26 18:34:17.200952 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:34:17.200919    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb82aa3e_e46e_4f8e_914e_fcfc67128310.slice/crio-df7c1b76d5bbdae2f28fe84d43cf9b19d8ab68fd173b0a18ecbedd515ae65e18 WatchSource:0}: Error finding container df7c1b76d5bbdae2f28fe84d43cf9b19d8ab68fd173b0a18ecbedd515ae65e18: Status 404 returned error can't find the container with id df7c1b76d5bbdae2f28fe84d43cf9b19d8ab68fd173b0a18ecbedd515ae65e18
Jul 26 18:34:17.248359 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.248333    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"]
Jul 26 18:34:17.250690 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:34:17.250662    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b26963_d067_40e3_b6f3_0ab5e88da2fa.slice/crio-c5c9abec75b49366bc64df5885290ead3a6625f2d5ea36926e375ae27149296d WatchSource:0}: Error finding container c5c9abec75b49366bc64df5885290ead3a6625f2d5ea36926e375ae27149296d: Status 404 returned error can't find the container with id c5c9abec75b49366bc64df5885290ead3a6625f2d5ea36926e375ae27149296d
Jul 26 18:34:17.598752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.598632    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84" event={"ID":"82b26963-d067-40e3-b6f3-0ab5e88da2fa","Type":"ContainerStarted","Data":"f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e"}
Jul 26 18:34:17.598752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.598714    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84" event={"ID":"82b26963-d067-40e3-b6f3-0ab5e88da2fa","Type":"ContainerStarted","Data":"c5c9abec75b49366bc64df5885290ead3a6625f2d5ea36926e375ae27149296d"}
Jul 26 18:34:17.599012 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.598851    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:17.600341 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.600315    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp" event={"ID":"eb82aa3e-e46e-4f8e-914e-fcfc67128310","Type":"ContainerStarted","Data":"1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87"}
Jul 26 18:34:17.600472 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.600354    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp" event={"ID":"eb82aa3e-e46e-4f8e-914e-fcfc67128310","Type":"ContainerStarted","Data":"df7c1b76d5bbdae2f28fe84d43cf9b19d8ab68fd173b0a18ecbedd515ae65e18"}
Jul 26 18:34:17.600472 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.600453    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:17.617448 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.617398    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84" podStartSLOduration=1.617382925 podStartE2EDuration="1.617382925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:17.615519046 +0000 UTC m=+2211.805548812" watchObservedRunningTime="2026-07-26 18:34:17.617382925 +0000 UTC m=+2211.807412678"
Jul 26 18:34:17.631311 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:17.631260    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp" podStartSLOduration=1.631243553 podStartE2EDuration="1.631243553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:17.630340525 +0000 UTC m=+2211.820370280" watchObservedRunningTime="2026-07-26 18:34:17.631243553 +0000 UTC m=+2211.821273307"
Jul 26 18:34:23.717316 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:23.717280    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:34:24.903667 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.903629    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"]
Jul 26 18:34:24.907150 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.907131    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:24.909969 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.909945    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-7d6ab3-dockercfg-qgfgt\""
Jul 26 18:34:24.910086 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.909965    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-7d6ab3\""
Jul 26 18:34:24.918102 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.918077    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"]
Jul 26 18:34:24.946787 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.946755    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"]
Jul 26 18:34:24.949895 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.949879    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:24.960902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.960877    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"]
Jul 26 18:34:24.998549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.998520    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60816894-5751-4d52-97d0-d9393be9988f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:24.998730 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:24.998565    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvsvq\" (UniqueName: \"kubernetes.io/projected/60816894-5751-4d52-97d0-d9393be9988f-kube-api-access-dvsvq\") pod \"payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.099195 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.099155    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60816894-5751-4d52-97d0-d9393be9988f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.099376 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.099211    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvsvq\" (UniqueName: \"kubernetes.io/projected/60816894-5751-4d52-97d0-d9393be9988f-kube-api-access-dvsvq\") pod \"payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.099376 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.099342    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-474b2\" (UniqueName: \"kubernetes.io/projected/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-kube-api-access-474b2\") pod \"payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.099468 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.099402    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.099953 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.099933    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60816894-5751-4d52-97d0-d9393be9988f-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.107845 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.107817    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvsvq\" (UniqueName: \"kubernetes.io/projected/60816894-5751-4d52-97d0-d9393be9988f-kube-api-access-dvsvq\") pod \"payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.200133 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.200040    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-474b2\" (UniqueName: \"kubernetes.io/projected/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-kube-api-access-474b2\") pod \"payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.200133 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.200091    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.200756 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.200736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.208733 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.208675    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-474b2\" (UniqueName: \"kubernetes.io/projected/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-kube-api-access-474b2\") pod \"payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.218386 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.218361    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.260851 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.260816    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.363546 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.363133    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"]
Jul 26 18:34:25.365527 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:34:25.365487    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60816894_5751_4d52_97d0_d9393be9988f.slice/crio-c4ed46855cf8fc56836109b7d77441179832d24e238fb12c170a5ab582e5e15d WatchSource:0}: Error finding container c4ed46855cf8fc56836109b7d77441179832d24e238fb12c170a5ab582e5e15d: Status 404 returned error can't find the container with id c4ed46855cf8fc56836109b7d77441179832d24e238fb12c170a5ab582e5e15d
Jul 26 18:34:25.411229 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.411205    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"]
Jul 26 18:34:25.415815 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:34:25.415789    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc15ef2_2ec8_41f4_96b4_75e7b572da50.slice/crio-f352bbb69027bae7501df035b99632608726694d6437dfe72ba9c79e69fd2ec2 WatchSource:0}: Error finding container f352bbb69027bae7501df035b99632608726694d6437dfe72ba9c79e69fd2ec2: Status 404 returned error can't find the container with id f352bbb69027bae7501df035b99632608726694d6437dfe72ba9c79e69fd2ec2
Jul 26 18:34:25.644088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.644056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr" event={"ID":"8dc15ef2-2ec8-41f4-96b4-75e7b572da50","Type":"ContainerStarted","Data":"f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283"}
Jul 26 18:34:25.644275 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.644108    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr" event={"ID":"8dc15ef2-2ec8-41f4-96b4-75e7b572da50","Type":"ContainerStarted","Data":"f352bbb69027bae7501df035b99632608726694d6437dfe72ba9c79e69fd2ec2"}
Jul 26 18:34:25.644275 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.644154    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:25.645576 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.645552    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr" event={"ID":"60816894-5751-4d52-97d0-d9393be9988f","Type":"ContainerStarted","Data":"f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab"}
Jul 26 18:34:25.645691 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.645599    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:25.645691 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.645619    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr" event={"ID":"60816894-5751-4d52-97d0-d9393be9988f","Type":"ContainerStarted","Data":"c4ed46855cf8fc56836109b7d77441179832d24e238fb12c170a5ab582e5e15d"}
Jul 26 18:34:25.661521 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.661467    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr" podStartSLOduration=1.661453035 podStartE2EDuration="1.661453035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:25.659683904 +0000 UTC m=+2219.849713674" watchObservedRunningTime="2026-07-26 18:34:25.661453035 +0000 UTC m=+2219.851482840"
Jul 26 18:34:25.678166 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:25.678117    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr" podStartSLOduration=1.6781011289999999 podStartE2EDuration="1.678101129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:25.676614413 +0000 UTC m=+2219.866644193" watchObservedRunningTime="2026-07-26 18:34:25.678101129 +0000 UTC m=+2219.868130946"
Jul 26 18:34:26.453933 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:26.453899    2565 scope.go:117] "RemoveContainer" containerID="5670c5e27511d63349d67d7cc824073e7b6346917c1c2fc14394aebaa632a27a"
Jul 26 18:34:28.607299 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:28.607257    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:34:28.607715 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:28.607481    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:34:31.033353 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.033318    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"]
Jul 26 18:34:31.036689 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.036665    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.039365 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.039337    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-872031\"/\"openshift-service-ca.crt\""
Jul 26 18:34:31.039479 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.039387    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-872031\"/\"default-dockercfg-5f426\""
Jul 26 18:34:31.039593 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.039574    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-872031\"/\"test-model-872031-kserve-self-signed-certs\""
Jul 26 18:34:31.039657 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.039625    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-872031\"/\"kube-root-ca.crt\""
Jul 26 18:34:31.046868 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.046847    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"]
Jul 26 18:34:31.151209 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.151169    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-home\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.151449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.151253    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa8b1172-272d-4a4a-968f-7a8989e093f4-tls-certs\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.151449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.151294    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-model-cache\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.151449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.151320    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtc96\" (UniqueName: \"kubernetes.io/projected/fa8b1172-272d-4a4a-968f-7a8989e093f4-kube-api-access-wtc96\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.151449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.151348    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-tmp-dir\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.151449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.151380    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-dshm\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252334 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252286    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa8b1172-272d-4a4a-968f-7a8989e093f4-tls-certs\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252334 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252335    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-model-cache\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252353    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtc96\" (UniqueName: \"kubernetes.io/projected/fa8b1172-272d-4a4a-968f-7a8989e093f4-kube-api-access-wtc96\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252414    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-tmp-dir\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252481    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-dshm\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252554    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-home\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252879 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252853    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-model-cache\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252935 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-tmp-dir\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.252935 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.252893    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-home\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.254819 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.254798    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-dshm\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.254995 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.254977    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa8b1172-272d-4a4a-968f-7a8989e093f4-tls-certs\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.260683 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.260663    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtc96\" (UniqueName: \"kubernetes.io/projected/fa8b1172-272d-4a4a-968f-7a8989e093f4-kube-api-access-wtc96\") pod \"test-model-872031-kserve-764f4c5ff-6jxv2\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") " pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.346763 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.346636    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:31.484262 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.484236    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"]
Jul 26 18:34:31.486043 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:34:31.486015    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa8b1172_272d_4a4a_968f_7a8989e093f4.slice/crio-0564ddf6cca7ab6ef8c2178540f9632c4207bfff4ece8e5fd26b27cd62b30498 WatchSource:0}: Error finding container 0564ddf6cca7ab6ef8c2178540f9632c4207bfff4ece8e5fd26b27cd62b30498: Status 404 returned error can't find the container with id 0564ddf6cca7ab6ef8c2178540f9632c4207bfff4ece8e5fd26b27cd62b30498
Jul 26 18:34:31.680065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.679975    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2" event={"ID":"fa8b1172-272d-4a4a-968f-7a8989e093f4","Type":"ContainerStarted","Data":"caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426"}
Jul 26 18:34:31.680065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.680023    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2" event={"ID":"fa8b1172-272d-4a4a-968f-7a8989e093f4","Type":"ContainerStarted","Data":"0564ddf6cca7ab6ef8c2178540f9632c4207bfff4ece8e5fd26b27cd62b30498"}
Jul 26 18:34:31.699345 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:31.699287    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2" podStartSLOduration=0.699260602 podStartE2EDuration="699.260602ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:31.697645095 +0000 UTC m=+2225.887674849" watchObservedRunningTime="2026-07-26 18:34:31.699260602 +0000 UTC m=+2225.889290373"
Jul 26 18:34:36.653901 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:36.653864    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:34:36.654390 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:36.654043    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:34:38.420170 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:38.420130    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:34:41.347027 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:41.346986    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:41.347560 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:41.347054    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:41.359903 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:41.359874    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:41.750396 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:41.750366    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:34:55.010507 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:34:55.010417    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:35:27.331309 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:27.331269    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"]
Jul 26 18:35:28.355595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.355562    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"]
Jul 26 18:35:28.359087 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.359070    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:28.361937 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.361915    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:35:28.361937 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.361915    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:35:28.366874 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.366852    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"]
Jul 26 18:35:28.496179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.496136    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebaf3e6-393f-449d-b3f5-519955b52e80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-872031-sq684\" (UID: \"eebaf3e6-393f-449d-b3f5-519955b52e80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:28.596952 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.596920    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebaf3e6-393f-449d-b3f5-519955b52e80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-872031-sq684\" (UID: \"eebaf3e6-393f-449d-b3f5-519955b52e80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:28.597545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.597523    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebaf3e6-393f-449d-b3f5-519955b52e80-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-872031-sq684\" (UID: \"eebaf3e6-393f-449d-b3f5-519955b52e80\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:28.670430 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.670323    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:28.800527 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.800504    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"]
Jul 26 18:35:28.802674 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:35:28.802649    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebaf3e6_393f_449d_b3f5_519955b52e80.slice/crio-3f86846580da40665cbedc1d49aa22eec763fb5d2e6252006960bc5ea06e7962 WatchSource:0}: Error finding container 3f86846580da40665cbedc1d49aa22eec763fb5d2e6252006960bc5ea06e7962: Status 404 returned error can't find the container with id 3f86846580da40665cbedc1d49aa22eec763fb5d2e6252006960bc5ea06e7962
Jul 26 18:35:28.973765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:28.973726    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684" event={"ID":"eebaf3e6-393f-449d-b3f5-519955b52e80","Type":"ContainerStarted","Data":"3f86846580da40665cbedc1d49aa22eec763fb5d2e6252006960bc5ea06e7962"}
Jul 26 18:35:29.980860 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:29.980823    2565 generic.go:356] "Generic (PLEG): container finished" podID="eebaf3e6-393f-449d-b3f5-519955b52e80" containerID="27a59db29039671dc7a6bc9ae506e667da60fdede00055ddfea42e7b4191a118" exitCode=0
Jul 26 18:35:29.981244 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:29.980899    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684" event={"ID":"eebaf3e6-393f-449d-b3f5-519955b52e80","Type":"ContainerDied","Data":"27a59db29039671dc7a6bc9ae506e667da60fdede00055ddfea42e7b4191a118"}
Jul 26 18:35:31.117924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.117902    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:31.220808 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.220769    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebaf3e6-393f-449d-b3f5-519955b52e80-maas-api-service-ca\") pod \"eebaf3e6-393f-449d-b3f5-519955b52e80\" (UID: \"eebaf3e6-393f-449d-b3f5-519955b52e80\") "
Jul 26 18:35:31.221180 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.221151    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebaf3e6-393f-449d-b3f5-519955b52e80-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eebaf3e6-393f-449d-b3f5-519955b52e80" (UID: "eebaf3e6-393f-449d-b3f5-519955b52e80"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:35:31.321682 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.321592    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eebaf3e6-393f-449d-b3f5-519955b52e80-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:31.991397 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.991367    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"
Jul 26 18:35:31.991581 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.991407    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684" event={"ID":"eebaf3e6-393f-449d-b3f5-519955b52e80","Type":"ContainerDied","Data":"3f86846580da40665cbedc1d49aa22eec763fb5d2e6252006960bc5ea06e7962"}
Jul 26 18:35:31.991581 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:31.991460    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f86846580da40665cbedc1d49aa22eec763fb5d2e6252006960bc5ea06e7962"
Jul 26 18:35:34.205931 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:34.205893    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:35:39.085646 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.085613    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"]
Jul 26 18:35:39.086207 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.085912    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84" podUID="82b26963-d067-40e3-b6f3-0ab5e88da2fa" containerName="payload-processing" containerID="cri-o://f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e" gracePeriod=30
Jul 26 18:35:39.093375 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.093341    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"]
Jul 26 18:35:39.093887 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.093575    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp" podUID="eb82aa3e-e46e-4f8e-914e-fcfc67128310" containerName="payload-pre-processing" containerID="cri-o://1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87" gracePeriod=30
Jul 26 18:35:39.350022 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.349999    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:35:39.377571 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.377543    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:35:39.489678 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.489639    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b26963-d067-40e3-b6f3-0ab5e88da2fa-plugins-config-volume\") pod \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") "
Jul 26 18:35:39.489868 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.489737    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnnwr\" (UniqueName: \"kubernetes.io/projected/eb82aa3e-e46e-4f8e-914e-fcfc67128310-kube-api-access-rnnwr\") pod \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") "
Jul 26 18:35:39.489868 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.489844    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlrzq\" (UniqueName: \"kubernetes.io/projected/82b26963-d067-40e3-b6f3-0ab5e88da2fa-kube-api-access-jlrzq\") pod \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\" (UID: \"82b26963-d067-40e3-b6f3-0ab5e88da2fa\") "
Jul 26 18:35:39.489963 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.489891    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb82aa3e-e46e-4f8e-914e-fcfc67128310-plugins-config-volume\") pod \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\" (UID: \"eb82aa3e-e46e-4f8e-914e-fcfc67128310\") "
Jul 26 18:35:39.490132 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.490102    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82b26963-d067-40e3-b6f3-0ab5e88da2fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82b26963-d067-40e3-b6f3-0ab5e88da2fa" (UID: "82b26963-d067-40e3-b6f3-0ab5e88da2fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:35:39.490289 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.490259    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb82aa3e-e46e-4f8e-914e-fcfc67128310-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb82aa3e-e46e-4f8e-914e-fcfc67128310" (UID: "eb82aa3e-e46e-4f8e-914e-fcfc67128310"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:35:39.491882 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.491858    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb82aa3e-e46e-4f8e-914e-fcfc67128310-kube-api-access-rnnwr" (OuterVolumeSpecName: "kube-api-access-rnnwr") pod "eb82aa3e-e46e-4f8e-914e-fcfc67128310" (UID: "eb82aa3e-e46e-4f8e-914e-fcfc67128310"). InnerVolumeSpecName "kube-api-access-rnnwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:35:39.491975 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.491910    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82b26963-d067-40e3-b6f3-0ab5e88da2fa-kube-api-access-jlrzq" (OuterVolumeSpecName: "kube-api-access-jlrzq") pod "82b26963-d067-40e3-b6f3-0ab5e88da2fa" (UID: "82b26963-d067-40e3-b6f3-0ab5e88da2fa"). InnerVolumeSpecName "kube-api-access-jlrzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:35:39.591025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.590942    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnnwr\" (UniqueName: \"kubernetes.io/projected/eb82aa3e-e46e-4f8e-914e-fcfc67128310-kube-api-access-rnnwr\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:39.591025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.590969    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlrzq\" (UniqueName: \"kubernetes.io/projected/82b26963-d067-40e3-b6f3-0ab5e88da2fa-kube-api-access-jlrzq\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:39.591025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.590978    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb82aa3e-e46e-4f8e-914e-fcfc67128310-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:39.591025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:39.590987    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82b26963-d067-40e3-b6f3-0ab5e88da2fa-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:40.036870 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.036835    2565 generic.go:356] "Generic (PLEG): container finished" podID="82b26963-d067-40e3-b6f3-0ab5e88da2fa" containerID="f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e" exitCode=0
Jul 26 18:35:40.037047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.036886    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"
Jul 26 18:35:40.037047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.036914    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84" event={"ID":"82b26963-d067-40e3-b6f3-0ab5e88da2fa","Type":"ContainerDied","Data":"f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e"}
Jul 26 18:35:40.037047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.036957    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84" event={"ID":"82b26963-d067-40e3-b6f3-0ab5e88da2fa","Type":"ContainerDied","Data":"c5c9abec75b49366bc64df5885290ead3a6625f2d5ea36926e375ae27149296d"}
Jul 26 18:35:40.037047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.036978    2565 scope.go:117] "RemoveContainer" containerID="f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e"
Jul 26 18:35:40.038412 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.038389    2565 generic.go:356] "Generic (PLEG): container finished" podID="eb82aa3e-e46e-4f8e-914e-fcfc67128310" containerID="1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87" exitCode=0
Jul 26 18:35:40.038513 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.038427    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"
Jul 26 18:35:40.038513 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.038449    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp" event={"ID":"eb82aa3e-e46e-4f8e-914e-fcfc67128310","Type":"ContainerDied","Data":"1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87"}
Jul 26 18:35:40.038513 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.038495    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp" event={"ID":"eb82aa3e-e46e-4f8e-914e-fcfc67128310","Type":"ContainerDied","Data":"df7c1b76d5bbdae2f28fe84d43cf9b19d8ab68fd173b0a18ecbedd515ae65e18"}
Jul 26 18:35:40.047342 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.047324    2565 scope.go:117] "RemoveContainer" containerID="f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e"
Jul 26 18:35:40.047648 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:35:40.047629    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e\": container with ID starting with f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e not found: ID does not exist" containerID="f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e"
Jul 26 18:35:40.047752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.047655    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e"} err="failed to get container status \"f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e\": rpc error: code = NotFound desc = could not find container \"f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e\": container with ID starting with f7fc98fdea7c151f1d8f4cf41775221ad6d0a2c2117d678a8a7e3d7cb2d79c5e not found: ID does not exist"
Jul 26 18:35:40.047752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.047671    2565 scope.go:117] "RemoveContainer" containerID="1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87"
Jul 26 18:35:40.055597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.055582    2565 scope.go:117] "RemoveContainer" containerID="1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87"
Jul 26 18:35:40.055872 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:35:40.055846    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87\": container with ID starting with 1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87 not found: ID does not exist" containerID="1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87"
Jul 26 18:35:40.055926 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.055881    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87"} err="failed to get container status \"1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87\": rpc error: code = NotFound desc = could not find container \"1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87\": container with ID starting with 1c25767fa71b3efd5b0adfec5d33345814aaf8ff5cbbb5335e82051cc8a48f87 not found: ID does not exist"
Jul 26 18:35:40.058608 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.058587    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"]
Jul 26 18:35:40.063557 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.063537    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-872031-56b85d8f6-8tc84"]
Jul 26 18:35:40.073531 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.073507    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"]
Jul 26 18:35:40.077128 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.077107    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-872031-754d8dd9d8-qm4tp"]
Jul 26 18:35:40.318860 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.318787    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82b26963-d067-40e3-b6f3-0ab5e88da2fa" path="/var/lib/kubelet/pods/82b26963-d067-40e3-b6f3-0ab5e88da2fa/volumes"
Jul 26 18:35:40.319204 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:40.319133    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb82aa3e-e46e-4f8e-914e-fcfc67128310" path="/var/lib/kubelet/pods/eb82aa3e-e46e-4f8e-914e-fcfc67128310/volumes"
Jul 26 18:35:42.350506 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.350464    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2" podUID="fa8b1172-272d-4a4a-968f-7a8989e093f4" containerName="main" containerID="cri-o://caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426" gracePeriod=45
Jul 26 18:35:42.591787 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.591763    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:35:42.718656 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718565    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-dshm\") pod \"fa8b1172-272d-4a4a-968f-7a8989e093f4\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") "
Jul 26 18:35:42.718656 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718610    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-model-cache\") pod \"fa8b1172-272d-4a4a-968f-7a8989e093f4\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") "
Jul 26 18:35:42.718923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718661    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtc96\" (UniqueName: \"kubernetes.io/projected/fa8b1172-272d-4a4a-968f-7a8989e093f4-kube-api-access-wtc96\") pod \"fa8b1172-272d-4a4a-968f-7a8989e093f4\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") "
Jul 26 18:35:42.718923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718681    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-tmp-dir\") pod \"fa8b1172-272d-4a4a-968f-7a8989e093f4\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") "
Jul 26 18:35:42.718923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718731    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa8b1172-272d-4a4a-968f-7a8989e093f4-tls-certs\") pod \"fa8b1172-272d-4a4a-968f-7a8989e093f4\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") "
Jul 26 18:35:42.718923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718871    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-home\") pod \"fa8b1172-272d-4a4a-968f-7a8989e093f4\" (UID: \"fa8b1172-272d-4a4a-968f-7a8989e093f4\") "
Jul 26 18:35:42.719150 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718984    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-model-cache" (OuterVolumeSpecName: "model-cache") pod "fa8b1172-272d-4a4a-968f-7a8989e093f4" (UID: "fa8b1172-272d-4a4a-968f-7a8989e093f4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:42.719150 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.718985    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fa8b1172-272d-4a4a-968f-7a8989e093f4" (UID: "fa8b1172-272d-4a4a-968f-7a8989e093f4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:42.719150 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.719127    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:42.719150 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.719145    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:42.719303 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.719139    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-home" (OuterVolumeSpecName: "home") pod "fa8b1172-272d-4a4a-968f-7a8989e093f4" (UID: "fa8b1172-272d-4a4a-968f-7a8989e093f4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:42.720939 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.720917    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-dshm" (OuterVolumeSpecName: "dshm") pod "fa8b1172-272d-4a4a-968f-7a8989e093f4" (UID: "fa8b1172-272d-4a4a-968f-7a8989e093f4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:42.721011 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.720950    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa8b1172-272d-4a4a-968f-7a8989e093f4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fa8b1172-272d-4a4a-968f-7a8989e093f4" (UID: "fa8b1172-272d-4a4a-968f-7a8989e093f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:35:42.721058 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.721022    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8b1172-272d-4a4a-968f-7a8989e093f4-kube-api-access-wtc96" (OuterVolumeSpecName: "kube-api-access-wtc96") pod "fa8b1172-272d-4a4a-968f-7a8989e093f4" (UID: "fa8b1172-272d-4a4a-968f-7a8989e093f4"). InnerVolumeSpecName "kube-api-access-wtc96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:35:42.820151 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.820115    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa8b1172-272d-4a4a-968f-7a8989e093f4-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:42.820151 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.820145    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:42.820151 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.820154    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa8b1172-272d-4a4a-968f-7a8989e093f4-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:42.820151 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:42.820162    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtc96\" (UniqueName: \"kubernetes.io/projected/fa8b1172-272d-4a4a-968f-7a8989e093f4-kube-api-access-wtc96\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:43.058624 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.058591    2565 generic.go:356] "Generic (PLEG): container finished" podID="fa8b1172-272d-4a4a-968f-7a8989e093f4" containerID="caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426" exitCode=0
Jul 26 18:35:43.058849 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.058635    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"
Jul 26 18:35:43.058849 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.058678    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2" event={"ID":"fa8b1172-272d-4a4a-968f-7a8989e093f4","Type":"ContainerDied","Data":"caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426"}
Jul 26 18:35:43.058849 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.058732    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2" event={"ID":"fa8b1172-272d-4a4a-968f-7a8989e093f4","Type":"ContainerDied","Data":"0564ddf6cca7ab6ef8c2178540f9632c4207bfff4ece8e5fd26b27cd62b30498"}
Jul 26 18:35:43.058849 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.058750    2565 scope.go:117] "RemoveContainer" containerID="caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426"
Jul 26 18:35:43.067771 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.067754    2565 scope.go:117] "RemoveContainer" containerID="caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426"
Jul 26 18:35:43.068029 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:35:43.068011    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426\": container with ID starting with caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426 not found: ID does not exist" containerID="caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426"
Jul 26 18:35:43.068075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.068038    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426"} err="failed to get container status \"caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426\": rpc error: code = NotFound desc = could not find container \"caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426\": container with ID starting with caf1f45d9b0642ddd2a4de9bbd910cdfc8b4de2fbe45914a19d184f29d43a426 not found: ID does not exist"
Jul 26 18:35:43.079905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.079873    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"]
Jul 26 18:35:43.083935 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.083912    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-872031/test-model-872031-kserve-764f4c5ff-6jxv2"]
Jul 26 18:35:43.445758 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.445661    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"]
Jul 26 18:35:43.447377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.447356    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-872031-sq684"]
Jul 26 18:35:43.721179 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:43.721147    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:35:44.318997 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:44.318961    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eebaf3e6-393f-449d-b3f5-519955b52e80" path="/var/lib/kubelet/pods/eebaf3e6-393f-449d-b3f5-519955b52e80/volumes"
Jul 26 18:35:44.319328 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:44.319317    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa8b1172-272d-4a4a-968f-7a8989e093f4" path="/var/lib/kubelet/pods/fa8b1172-272d-4a4a-968f-7a8989e093f4/volumes"
Jul 26 18:35:50.258753 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.258722    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"]
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259016    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82b26963-d067-40e3-b6f3-0ab5e88da2fa" containerName="payload-processing"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259027    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="82b26963-d067-40e3-b6f3-0ab5e88da2fa" containerName="payload-processing"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259036    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa8b1172-272d-4a4a-968f-7a8989e093f4" containerName="main"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259041    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa8b1172-272d-4a4a-968f-7a8989e093f4" containerName="main"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259055    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eebaf3e6-393f-449d-b3f5-519955b52e80" containerName="revoke-keys"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259063    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebaf3e6-393f-449d-b3f5-519955b52e80" containerName="revoke-keys"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259077    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb82aa3e-e46e-4f8e-914e-fcfc67128310" containerName="payload-pre-processing"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259082    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb82aa3e-e46e-4f8e-914e-fcfc67128310" containerName="payload-pre-processing"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259126    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb82aa3e-e46e-4f8e-914e-fcfc67128310" containerName="payload-pre-processing"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259134    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="82b26963-d067-40e3-b6f3-0ab5e88da2fa" containerName="payload-processing"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259140    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="eebaf3e6-393f-449d-b3f5-519955b52e80" containerName="revoke-keys"
Jul 26 18:35:50.259139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.259147    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa8b1172-272d-4a4a-968f-7a8989e093f4" containerName="main"
Jul 26 18:35:50.263268 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.263250    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:50.266191 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.266172    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:35:50.266326 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.266196    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:35:50.271444 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.271413    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"]
Jul 26 18:35:50.381681 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.381646    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3be138-3769-412e-acee-0ef042386176-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj\" (UID: \"5e3be138-3769-412e-acee-0ef042386176\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:50.482889 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.482853    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3be138-3769-412e-acee-0ef042386176-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj\" (UID: \"5e3be138-3769-412e-acee-0ef042386176\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:50.483499 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.483476    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3be138-3769-412e-acee-0ef042386176-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj\" (UID: \"5e3be138-3769-412e-acee-0ef042386176\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:50.574548 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.574447    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:50.704317 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:50.704240    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"]
Jul 26 18:35:50.706470 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:35:50.706442    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3be138_3769_412e_acee_0ef042386176.slice/crio-6c3f2ff343093b54483ae859800b2eb312d3c2013d3e4fa9f51b4446f4e78bf5 WatchSource:0}: Error finding container 6c3f2ff343093b54483ae859800b2eb312d3c2013d3e4fa9f51b4446f4e78bf5: Status 404 returned error can't find the container with id 6c3f2ff343093b54483ae859800b2eb312d3c2013d3e4fa9f51b4446f4e78bf5
Jul 26 18:35:51.101125 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:51.101091    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj" event={"ID":"5e3be138-3769-412e-acee-0ef042386176","Type":"ContainerStarted","Data":"24fd9de890a6b1dc02034e4eb6511d89319b5d7ebeef3b70224107da62298225"}
Jul 26 18:35:51.101308 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:51.101142    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj" event={"ID":"5e3be138-3769-412e-acee-0ef042386176","Type":"ContainerStarted","Data":"6c3f2ff343093b54483ae859800b2eb312d3c2013d3e4fa9f51b4446f4e78bf5"}
Jul 26 18:35:51.117128 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:51.117078    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj" podStartSLOduration=1.117063126 podStartE2EDuration="1.117063126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:35:51.116867291 +0000 UTC m=+2305.306897045" watchObservedRunningTime="2026-07-26 18:35:51.117063126 +0000 UTC m=+2305.307092879"
Jul 26 18:35:52.107243 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:52.107208    2565 generic.go:356] "Generic (PLEG): container finished" podID="5e3be138-3769-412e-acee-0ef042386176" containerID="24fd9de890a6b1dc02034e4eb6511d89319b5d7ebeef3b70224107da62298225" exitCode=0
Jul 26 18:35:52.107625 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:52.107296    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj" event={"ID":"5e3be138-3769-412e-acee-0ef042386176","Type":"ContainerDied","Data":"24fd9de890a6b1dc02034e4eb6511d89319b5d7ebeef3b70224107da62298225"}
Jul 26 18:35:53.239711 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:53.239674    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:53.407139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:53.407049    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3be138-3769-412e-acee-0ef042386176-maas-api-service-ca\") pod \"5e3be138-3769-412e-acee-0ef042386176\" (UID: \"5e3be138-3769-412e-acee-0ef042386176\") "
Jul 26 18:35:53.407452 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:53.407420    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e3be138-3769-412e-acee-0ef042386176-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5e3be138-3769-412e-acee-0ef042386176" (UID: "5e3be138-3769-412e-acee-0ef042386176"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:35:53.508044 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:53.508011    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e3be138-3769-412e-acee-0ef042386176-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:35:54.118143 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:54.118115    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"
Jul 26 18:35:54.118143 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:54.118136    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj" event={"ID":"5e3be138-3769-412e-acee-0ef042386176","Type":"ContainerDied","Data":"6c3f2ff343093b54483ae859800b2eb312d3c2013d3e4fa9f51b4446f4e78bf5"}
Jul 26 18:35:54.118353 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:35:54.118181    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6c3f2ff343093b54483ae859800b2eb312d3c2013d3e4fa9f51b4446f4e78bf5"
Jul 26 18:36:00.749265 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:00.749229    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"]
Jul 26 18:36:00.749900 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:00.749462    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr" podUID="60816894-5751-4d52-97d0-d9393be9988f" containerName="payload-processing" containerID="cri-o://f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab" gracePeriod=30
Jul 26 18:36:00.756622 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:00.756593    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"]
Jul 26 18:36:00.757685 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:00.757653    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr" podUID="8dc15ef2-2ec8-41f4-96b4-75e7b572da50" containerName="payload-pre-processing" containerID="cri-o://f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283" gracePeriod=30
Jul 26 18:36:01.033119 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.033091    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:36:01.035780 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.035717    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:36:01.156015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.155971    2565 generic.go:356] "Generic (PLEG): container finished" podID="8dc15ef2-2ec8-41f4-96b4-75e7b572da50" containerID="f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283" exitCode=0
Jul 26 18:36:01.156015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.156017    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"
Jul 26 18:36:01.156277 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.156057    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr" event={"ID":"8dc15ef2-2ec8-41f4-96b4-75e7b572da50","Type":"ContainerDied","Data":"f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283"}
Jul 26 18:36:01.156277 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.156103    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr" event={"ID":"8dc15ef2-2ec8-41f4-96b4-75e7b572da50","Type":"ContainerDied","Data":"f352bbb69027bae7501df035b99632608726694d6437dfe72ba9c79e69fd2ec2"}
Jul 26 18:36:01.156277 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.156131    2565 scope.go:117] "RemoveContainer" containerID="f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283"
Jul 26 18:36:01.157262 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.157243    2565 generic.go:356] "Generic (PLEG): container finished" podID="60816894-5751-4d52-97d0-d9393be9988f" containerID="f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab" exitCode=0
Jul 26 18:36:01.157373 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.157284    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"
Jul 26 18:36:01.157373 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.157309    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr" event={"ID":"60816894-5751-4d52-97d0-d9393be9988f","Type":"ContainerDied","Data":"f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab"}
Jul 26 18:36:01.157373 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.157339    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr" event={"ID":"60816894-5751-4d52-97d0-d9393be9988f","Type":"ContainerDied","Data":"c4ed46855cf8fc56836109b7d77441179832d24e238fb12c170a5ab582e5e15d"}
Jul 26 18:36:01.164631 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.164616    2565 scope.go:117] "RemoveContainer" containerID="f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283"
Jul 26 18:36:01.164902 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:36:01.164884    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283\": container with ID starting with f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283 not found: ID does not exist" containerID="f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283"
Jul 26 18:36:01.164958 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.164910    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283"} err="failed to get container status \"f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283\": rpc error: code = NotFound desc = could not find container \"f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283\": container with ID starting with f6098397042dd6769cd305a02dfcab01cc5e6e18ce9aad7841938a145adaa283 not found: ID does not exist"
Jul 26 18:36:01.164958 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.164926    2565 scope.go:117] "RemoveContainer" containerID="f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab"
Jul 26 18:36:01.173038 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.173021    2565 scope.go:117] "RemoveContainer" containerID="f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab"
Jul 26 18:36:01.173307 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:36:01.173290    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab\": container with ID starting with f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab not found: ID does not exist" containerID="f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab"
Jul 26 18:36:01.173365 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.173313    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab"} err="failed to get container status \"f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab\": rpc error: code = NotFound desc = could not find container \"f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab\": container with ID starting with f433846744f13e1cac3af32dc203aebe4b82daaf8c0eba9fe13d9ea0dabbeeab not found: ID does not exist"
Jul 26 18:36:01.175570 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.175551    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-plugins-config-volume\") pod \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") "
Jul 26 18:36:01.175628 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.175589    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvsvq\" (UniqueName: \"kubernetes.io/projected/60816894-5751-4d52-97d0-d9393be9988f-kube-api-access-dvsvq\") pod \"60816894-5751-4d52-97d0-d9393be9988f\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") "
Jul 26 18:36:01.175628 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.175619    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60816894-5751-4d52-97d0-d9393be9988f-plugins-config-volume\") pod \"60816894-5751-4d52-97d0-d9393be9988f\" (UID: \"60816894-5751-4d52-97d0-d9393be9988f\") "
Jul 26 18:36:01.175758 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.175674    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-474b2\" (UniqueName: \"kubernetes.io/projected/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-kube-api-access-474b2\") pod \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\" (UID: \"8dc15ef2-2ec8-41f4-96b4-75e7b572da50\") "
Jul 26 18:36:01.175977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.175952    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8dc15ef2-2ec8-41f4-96b4-75e7b572da50" (UID: "8dc15ef2-2ec8-41f4-96b4-75e7b572da50"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:36:01.176046 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.175953    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60816894-5751-4d52-97d0-d9393be9988f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "60816894-5751-4d52-97d0-d9393be9988f" (UID: "60816894-5751-4d52-97d0-d9393be9988f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:36:01.177529 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.177508    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60816894-5751-4d52-97d0-d9393be9988f-kube-api-access-dvsvq" (OuterVolumeSpecName: "kube-api-access-dvsvq") pod "60816894-5751-4d52-97d0-d9393be9988f" (UID: "60816894-5751-4d52-97d0-d9393be9988f"). InnerVolumeSpecName "kube-api-access-dvsvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:36:01.177637 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.177620    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-kube-api-access-474b2" (OuterVolumeSpecName: "kube-api-access-474b2") pod "8dc15ef2-2ec8-41f4-96b4-75e7b572da50" (UID: "8dc15ef2-2ec8-41f4-96b4-75e7b572da50"). InnerVolumeSpecName "kube-api-access-474b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:36:01.276514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.276429    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60816894-5751-4d52-97d0-d9393be9988f-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:01.276514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.276458    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-474b2\" (UniqueName: \"kubernetes.io/projected/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-kube-api-access-474b2\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:01.276514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.276469    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dc15ef2-2ec8-41f4-96b4-75e7b572da50-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:01.276514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.276478    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvsvq\" (UniqueName: \"kubernetes.io/projected/60816894-5751-4d52-97d0-d9393be9988f-kube-api-access-dvsvq\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:01.483381 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.483344    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"]
Jul 26 18:36:01.487415 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.487387    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7d6ab3-6b88599958-7kjhr"]
Jul 26 18:36:01.498080 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.498050    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"]
Jul 26 18:36:01.504257 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:01.504232    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7d6ab3-844b5bcb84-c8xfr"]
Jul 26 18:36:02.318333 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:02.318297    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60816894-5751-4d52-97d0-d9393be9988f" path="/var/lib/kubelet/pods/60816894-5751-4d52-97d0-d9393be9988f/volumes"
Jul 26 18:36:02.318719 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:02.318627    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dc15ef2-2ec8-41f4-96b4-75e7b572da50" path="/var/lib/kubelet/pods/8dc15ef2-2ec8-41f4-96b4-75e7b572da50/volumes"
Jul 26 18:36:03.614458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:03.614421    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:36:05.328649 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:05.328613    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"]
Jul 26 18:36:05.331227 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:05.331199    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7d6ab3-2zkjj"]
Jul 26 18:36:06.319894 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:06.319862    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e3be138-3769-412e-acee-0ef042386176" path="/var/lib/kubelet/pods/5e3be138-3769-412e-acee-0ef042386176/volumes"
Jul 26 18:36:16.543423 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543389    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"]
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543667    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60816894-5751-4d52-97d0-d9393be9988f" containerName="payload-processing"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543677    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="60816894-5751-4d52-97d0-d9393be9988f" containerName="payload-processing"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543685    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dc15ef2-2ec8-41f4-96b4-75e7b572da50" containerName="payload-pre-processing"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543692    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dc15ef2-2ec8-41f4-96b4-75e7b572da50" containerName="payload-pre-processing"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543722    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e3be138-3769-412e-acee-0ef042386176" containerName="revoke-keys"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543729    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e3be138-3769-412e-acee-0ef042386176" containerName="revoke-keys"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543796    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="60816894-5751-4d52-97d0-d9393be9988f" containerName="payload-processing"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543806    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e3be138-3769-412e-acee-0ef042386176" containerName="revoke-keys"
Jul 26 18:36:16.543830 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.543811    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dc15ef2-2ec8-41f4-96b4-75e7b572da50" containerName="payload-pre-processing"
Jul 26 18:36:16.547765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.547739    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.550480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.550456    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-138d4e\""
Jul 26 18:36:16.557260 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.557234    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"]
Jul 26 18:36:16.590371 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.590331    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"]
Jul 26 18:36:16.598115 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.598083    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkdkk\" (UniqueName: \"kubernetes.io/projected/706051b6-5e60-4965-a013-86f0b1d0700a-kube-api-access-vkdkk\") pod \"payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.598267 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.598128    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706051b6-5e60-4965-a013-86f0b1d0700a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.599400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.599382    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.602300 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.602275    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-138d4e-dockercfg-2jqtk\""
Jul 26 18:36:16.603647 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.603621    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"]
Jul 26 18:36:16.698612 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.698579    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvzh5\" (UniqueName: \"kubernetes.io/projected/3fd65d93-e152-475d-b97e-521e341227bc-kube-api-access-nvzh5\") pod \"payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") " pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.698612 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.698617    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fd65d93-e152-475d-b97e-521e341227bc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") " pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.698879 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.698644    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkdkk\" (UniqueName: \"kubernetes.io/projected/706051b6-5e60-4965-a013-86f0b1d0700a-kube-api-access-vkdkk\") pod \"payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.698879 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.698731    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706051b6-5e60-4965-a013-86f0b1d0700a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.699398 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.699378    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706051b6-5e60-4965-a013-86f0b1d0700a-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.707471 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.707445    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkdkk\" (UniqueName: \"kubernetes.io/projected/706051b6-5e60-4965-a013-86f0b1d0700a-kube-api-access-vkdkk\") pod \"payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.800086 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.799997    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvzh5\" (UniqueName: \"kubernetes.io/projected/3fd65d93-e152-475d-b97e-521e341227bc-kube-api-access-nvzh5\") pod \"payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") " pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.800086 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.800035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fd65d93-e152-475d-b97e-521e341227bc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") " pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.800734 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.800687    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fd65d93-e152-475d-b97e-521e341227bc-plugins-config-volume\") pod \"payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") " pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.807950 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.807923    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvzh5\" (UniqueName: \"kubernetes.io/projected/3fd65d93-e152-475d-b97e-521e341227bc-kube-api-access-nvzh5\") pod \"payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") " pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:16.858159 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.858131    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:16.911184 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:16.911055    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:17.003044 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.003014    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"]
Jul 26 18:36:17.006544 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:36:17.006500    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706051b6_5e60_4965_a013_86f0b1d0700a.slice/crio-a587b26cf022399ba0bc25c4cb28e1ec4b666737d801477e5013f86236b65558 WatchSource:0}: Error finding container a587b26cf022399ba0bc25c4cb28e1ec4b666737d801477e5013f86236b65558: Status 404 returned error can't find the container with id a587b26cf022399ba0bc25c4cb28e1ec4b666737d801477e5013f86236b65558
Jul 26 18:36:17.065274 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.065249    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"]
Jul 26 18:36:17.067368 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:36:17.067332    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd65d93_e152_475d_b97e_521e341227bc.slice/crio-5780d1b42cdf0ff5e892e972a58077a0ae89afe447c2d6f61afb2315ac8059c0 WatchSource:0}: Error finding container 5780d1b42cdf0ff5e892e972a58077a0ae89afe447c2d6f61afb2315ac8059c0: Status 404 returned error can't find the container with id 5780d1b42cdf0ff5e892e972a58077a0ae89afe447c2d6f61afb2315ac8059c0
Jul 26 18:36:17.243826 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.243791    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d" event={"ID":"3fd65d93-e152-475d-b97e-521e341227bc","Type":"ContainerStarted","Data":"6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065"}
Jul 26 18:36:17.244035 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.243850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d" event={"ID":"3fd65d93-e152-475d-b97e-521e341227bc","Type":"ContainerStarted","Data":"5780d1b42cdf0ff5e892e972a58077a0ae89afe447c2d6f61afb2315ac8059c0"}
Jul 26 18:36:17.244035 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.243890    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:17.245364 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.245341    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz" event={"ID":"706051b6-5e60-4965-a013-86f0b1d0700a","Type":"ContainerStarted","Data":"0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d"}
Jul 26 18:36:17.245476 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.245384    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz" event={"ID":"706051b6-5e60-4965-a013-86f0b1d0700a","Type":"ContainerStarted","Data":"a587b26cf022399ba0bc25c4cb28e1ec4b666737d801477e5013f86236b65558"}
Jul 26 18:36:17.245476 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.245412    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:17.261406 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.261364    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d" podStartSLOduration=1.261350625 podStartE2EDuration="1.261350625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:36:17.259967682 +0000 UTC m=+2331.449997440" watchObservedRunningTime="2026-07-26 18:36:17.261350625 +0000 UTC m=+2331.451380356"
Jul 26 18:36:17.287455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:17.287393    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz" podStartSLOduration=1.287372589 podStartE2EDuration="1.287372589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:36:17.285227839 +0000 UTC m=+2331.475257589" watchObservedRunningTime="2026-07-26 18:36:17.287372589 +0000 UTC m=+2331.477402339"
Jul 26 18:36:23.621349 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:23.621255    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:36:24.707775 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.707745    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"]
Jul 26 18:36:24.711143 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.711126    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.713922 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.713892    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f349a2\""
Jul 26 18:36:24.714054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.713954    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f349a2-dockercfg-gs2qv\""
Jul 26 18:36:24.721212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.721190    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"]
Jul 26 18:36:24.764759 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.764718    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"]
Jul 26 18:36:24.764996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.764974    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a3a315-d552-4f36-b773-0874361e7e90-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") " pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.765048 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.765023    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lx8l\" (UniqueName: \"kubernetes.io/projected/32a3a315-d552-4f36-b773-0874361e7e90-kube-api-access-8lx8l\") pod \"payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") " pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.767969 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.767952    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:24.777321 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.777294    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"]
Jul 26 18:36:24.865791 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.865741    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a3a315-d552-4f36-b773-0874361e7e90-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") " pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.865791 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.865779    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlql4\" (UniqueName: \"kubernetes.io/projected/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-kube-api-access-dlql4\") pod \"payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:24.866010 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.865820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lx8l\" (UniqueName: \"kubernetes.io/projected/32a3a315-d552-4f36-b773-0874361e7e90-kube-api-access-8lx8l\") pod \"payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") " pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.866010 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.865842    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:24.866419 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.866398    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a3a315-d552-4f36-b773-0874361e7e90-plugins-config-volume\") pod \"payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") " pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.873893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.873867    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lx8l\" (UniqueName: \"kubernetes.io/projected/32a3a315-d552-4f36-b773-0874361e7e90-kube-api-access-8lx8l\") pod \"payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") " pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:24.967117 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.967027    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlql4\" (UniqueName: \"kubernetes.io/projected/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-kube-api-access-dlql4\") pod \"payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:24.967117 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.967087    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:24.967817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.967795    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:24.975139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:24.975107    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlql4\" (UniqueName: \"kubernetes.io/projected/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-kube-api-access-dlql4\") pod \"payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:25.022033 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.022000    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:25.078865 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.078838    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:25.189950 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.189920    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"]
Jul 26 18:36:25.191834 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:36:25.191804    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a3a315_d552_4f36_b773_0874361e7e90.slice/crio-47c7bd189951d568a579bff0c8f9f527ce642bdd82e88055e47399920b2083f9 WatchSource:0}: Error finding container 47c7bd189951d568a579bff0c8f9f527ce642bdd82e88055e47399920b2083f9: Status 404 returned error can't find the container with id 47c7bd189951d568a579bff0c8f9f527ce642bdd82e88055e47399920b2083f9
Jul 26 18:36:25.246720 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:36:25.246675    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51ff061_9ce9_46f6_ba6c_c6c95bdd8917.slice/crio-b841a79034a995ce820a25bb810782281e7f4cf723040a3b316de364e6e43e4d WatchSource:0}: Error finding container b841a79034a995ce820a25bb810782281e7f4cf723040a3b316de364e6e43e4d: Status 404 returned error can't find the container with id b841a79034a995ce820a25bb810782281e7f4cf723040a3b316de364e6e43e4d
Jul 26 18:36:25.247874 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.247847    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"]
Jul 26 18:36:25.286758 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.286729    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8" event={"ID":"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917","Type":"ContainerStarted","Data":"b841a79034a995ce820a25bb810782281e7f4cf723040a3b316de364e6e43e4d"}
Jul 26 18:36:25.288498 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.288456    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k" event={"ID":"32a3a315-d552-4f36-b773-0874361e7e90","Type":"ContainerStarted","Data":"f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab"}
Jul 26 18:36:25.288622 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.288512    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k" event={"ID":"32a3a315-d552-4f36-b773-0874361e7e90","Type":"ContainerStarted","Data":"47c7bd189951d568a579bff0c8f9f527ce642bdd82e88055e47399920b2083f9"}
Jul 26 18:36:25.288622 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.288598    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:25.320025 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:25.319966    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k" podStartSLOduration=1.319945158 podStartE2EDuration="1.319945158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:36:25.31862394 +0000 UTC m=+2339.508653695" watchObservedRunningTime="2026-07-26 18:36:25.319945158 +0000 UTC m=+2339.509974946"
Jul 26 18:36:26.295711 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:26.295669    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8" event={"ID":"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917","Type":"ContainerStarted","Data":"27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588"}
Jul 26 18:36:26.316488 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:26.316429    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8" podStartSLOduration=2.316409848 podStartE2EDuration="2.316409848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:36:26.313920086 +0000 UTC m=+2340.503949839" watchObservedRunningTime="2026-07-26 18:36:26.316409848 +0000 UTC m=+2340.506439603"
Jul 26 18:36:26.482382 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:26.482349    2565 scope.go:117] "RemoveContainer" containerID="51958bb54dbf90d2fa5c1a2242047beb730f91f0936d4a3fd6186c47bccb34f3"
Jul 26 18:36:27.300334 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:27.300304    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:28.251775 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:28.251745    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:28.251944 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:28.251818    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:36.047180 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.047147    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"]
Jul 26 18:36:36.050371 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.050353    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:36.052975 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.052950    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:36:36.053111 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.053055    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:36:36.058828 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.058805    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"]
Jul 26 18:36:36.167265 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.167228    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/292aace2-6c74-4755-994e-783b41e628bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-138d4e-sts68\" (UID: \"292aace2-6c74-4755-994e-783b41e628bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:36.267964 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.267930    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/292aace2-6c74-4755-994e-783b41e628bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-138d4e-sts68\" (UID: \"292aace2-6c74-4755-994e-783b41e628bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:36.268534 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.268513    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/292aace2-6c74-4755-994e-783b41e628bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-138d4e-sts68\" (UID: \"292aace2-6c74-4755-994e-783b41e628bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:36.298562 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.298490    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:36:36.361516 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.361484    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:36.491952 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:36.491927    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"]
Jul 26 18:36:36.494019 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:36:36.493996    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292aace2_6c74_4755_994e_783b41e628bf.slice/crio-4be35e984ed6d4deb2a862768aec3ab69233353fc2b8245cc7375919b6be63c7 WatchSource:0}: Error finding container 4be35e984ed6d4deb2a862768aec3ab69233353fc2b8245cc7375919b6be63c7: Status 404 returned error can't find the container with id 4be35e984ed6d4deb2a862768aec3ab69233353fc2b8245cc7375919b6be63c7
Jul 26 18:36:37.352505 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:37.352474    2565 generic.go:356] "Generic (PLEG): container finished" podID="292aace2-6c74-4755-994e-783b41e628bf" containerID="ef6a38b3deeac646c884a053d7b83c352bbea88aefa98d552dccfbae4d626cec" exitCode=0
Jul 26 18:36:37.352921 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:37.352558    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68" event={"ID":"292aace2-6c74-4755-994e-783b41e628bf","Type":"ContainerDied","Data":"ef6a38b3deeac646c884a053d7b83c352bbea88aefa98d552dccfbae4d626cec"}
Jul 26 18:36:37.352921 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:37.352613    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68" event={"ID":"292aace2-6c74-4755-994e-783b41e628bf","Type":"ContainerStarted","Data":"4be35e984ed6d4deb2a862768aec3ab69233353fc2b8245cc7375919b6be63c7"}
Jul 26 18:36:38.309321 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:38.309296    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:36:38.492043 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:38.492011    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:38.587760 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:38.587627    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/292aace2-6c74-4755-994e-783b41e628bf-maas-api-service-ca\") pod \"292aace2-6c74-4755-994e-783b41e628bf\" (UID: \"292aace2-6c74-4755-994e-783b41e628bf\") "
Jul 26 18:36:38.588026 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:38.588003    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/292aace2-6c74-4755-994e-783b41e628bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "292aace2-6c74-4755-994e-783b41e628bf" (UID: "292aace2-6c74-4755-994e-783b41e628bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:36:38.689054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:38.689016    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/292aace2-6c74-4755-994e-783b41e628bf-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:39.364592 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:39.364557    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68" event={"ID":"292aace2-6c74-4755-994e-783b41e628bf","Type":"ContainerDied","Data":"4be35e984ed6d4deb2a862768aec3ab69233353fc2b8245cc7375919b6be63c7"}
Jul 26 18:36:39.364592 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:39.364590    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"
Jul 26 18:36:39.364827 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:39.364606    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4be35e984ed6d4deb2a862768aec3ab69233353fc2b8245cc7375919b6be63c7"
Jul 26 18:36:41.409498 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:41.409467    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:36:52.514618 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.514584    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"]
Jul 26 18:36:52.515117 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.514836    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d" podUID="3fd65d93-e152-475d-b97e-521e341227bc" containerName="payload-processing" containerID="cri-o://6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065" gracePeriod=30
Jul 26 18:36:52.519904 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.519874    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"]
Jul 26 18:36:52.520132 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.520106    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz" podUID="706051b6-5e60-4965-a013-86f0b1d0700a" containerName="payload-pre-processing" containerID="cri-o://0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d" gracePeriod=30
Jul 26 18:36:52.802753 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.802722    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:52.805414 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.805394    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:52.915540 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.915497    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkdkk\" (UniqueName: \"kubernetes.io/projected/706051b6-5e60-4965-a013-86f0b1d0700a-kube-api-access-vkdkk\") pod \"706051b6-5e60-4965-a013-86f0b1d0700a\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") "
Jul 26 18:36:52.915765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.915588    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fd65d93-e152-475d-b97e-521e341227bc-plugins-config-volume\") pod \"3fd65d93-e152-475d-b97e-521e341227bc\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") "
Jul 26 18:36:52.915765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.915645    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706051b6-5e60-4965-a013-86f0b1d0700a-plugins-config-volume\") pod \"706051b6-5e60-4965-a013-86f0b1d0700a\" (UID: \"706051b6-5e60-4965-a013-86f0b1d0700a\") "
Jul 26 18:36:52.915765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.915686    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvzh5\" (UniqueName: \"kubernetes.io/projected/3fd65d93-e152-475d-b97e-521e341227bc-kube-api-access-nvzh5\") pod \"3fd65d93-e152-475d-b97e-521e341227bc\" (UID: \"3fd65d93-e152-475d-b97e-521e341227bc\") "
Jul 26 18:36:52.916046 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.916020    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fd65d93-e152-475d-b97e-521e341227bc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3fd65d93-e152-475d-b97e-521e341227bc" (UID: "3fd65d93-e152-475d-b97e-521e341227bc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:36:52.916103 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.916056    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706051b6-5e60-4965-a013-86f0b1d0700a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "706051b6-5e60-4965-a013-86f0b1d0700a" (UID: "706051b6-5e60-4965-a013-86f0b1d0700a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:36:52.917760 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.917738    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/706051b6-5e60-4965-a013-86f0b1d0700a-kube-api-access-vkdkk" (OuterVolumeSpecName: "kube-api-access-vkdkk") pod "706051b6-5e60-4965-a013-86f0b1d0700a" (UID: "706051b6-5e60-4965-a013-86f0b1d0700a"). InnerVolumeSpecName "kube-api-access-vkdkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:36:52.917867 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:52.917771    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fd65d93-e152-475d-b97e-521e341227bc-kube-api-access-nvzh5" (OuterVolumeSpecName: "kube-api-access-nvzh5") pod "3fd65d93-e152-475d-b97e-521e341227bc" (UID: "3fd65d93-e152-475d-b97e-521e341227bc"). InnerVolumeSpecName "kube-api-access-nvzh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:36:53.017066 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.017027    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/706051b6-5e60-4965-a013-86f0b1d0700a-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:53.017066 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.017060    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvzh5\" (UniqueName: \"kubernetes.io/projected/3fd65d93-e152-475d-b97e-521e341227bc-kube-api-access-nvzh5\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:53.017066 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.017071    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkdkk\" (UniqueName: \"kubernetes.io/projected/706051b6-5e60-4965-a013-86f0b1d0700a-kube-api-access-vkdkk\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:53.017317 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.017080    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fd65d93-e152-475d-b97e-521e341227bc-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:36:53.437673 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.437636    2565 generic.go:356] "Generic (PLEG): container finished" podID="3fd65d93-e152-475d-b97e-521e341227bc" containerID="6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065" exitCode=0
Jul 26 18:36:53.437893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.437684    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"
Jul 26 18:36:53.437893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.437733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d" event={"ID":"3fd65d93-e152-475d-b97e-521e341227bc","Type":"ContainerDied","Data":"6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065"}
Jul 26 18:36:53.437893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.437796    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d" event={"ID":"3fd65d93-e152-475d-b97e-521e341227bc","Type":"ContainerDied","Data":"5780d1b42cdf0ff5e892e972a58077a0ae89afe447c2d6f61afb2315ac8059c0"}
Jul 26 18:36:53.437893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.437822    2565 scope.go:117] "RemoveContainer" containerID="6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065"
Jul 26 18:36:53.438979 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.438960    2565 generic.go:356] "Generic (PLEG): container finished" podID="706051b6-5e60-4965-a013-86f0b1d0700a" containerID="0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d" exitCode=0
Jul 26 18:36:53.439065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.439003    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz" event={"ID":"706051b6-5e60-4965-a013-86f0b1d0700a","Type":"ContainerDied","Data":"0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d"}
Jul 26 18:36:53.439065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.439006    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"
Jul 26 18:36:53.439065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.439039    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz" event={"ID":"706051b6-5e60-4965-a013-86f0b1d0700a","Type":"ContainerDied","Data":"a587b26cf022399ba0bc25c4cb28e1ec4b666737d801477e5013f86236b65558"}
Jul 26 18:36:53.448908 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.448887    2565 scope.go:117] "RemoveContainer" containerID="6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065"
Jul 26 18:36:53.449389 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:36:53.449359    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065\": container with ID starting with 6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065 not found: ID does not exist" containerID="6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065"
Jul 26 18:36:53.449462 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.449405    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065"} err="failed to get container status \"6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065\": rpc error: code = NotFound desc = could not find container \"6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065\": container with ID starting with 6b5f520efd2706f5bb0c9fcaa04a17e2c71e95254cc077982d48f00dabc85065 not found: ID does not exist"
Jul 26 18:36:53.449462 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.449432    2565 scope.go:117] "RemoveContainer" containerID="0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d"
Jul 26 18:36:53.457377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.457361    2565 scope.go:117] "RemoveContainer" containerID="0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d"
Jul 26 18:36:53.457591 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:36:53.457575    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d\": container with ID starting with 0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d not found: ID does not exist" containerID="0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d"
Jul 26 18:36:53.457654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.457599    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d"} err="failed to get container status \"0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d\": rpc error: code = NotFound desc = could not find container \"0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d\": container with ID starting with 0ac8d9efd3b957f96255f1b2b183fec5c7fb56dd0c2a72a67575c8a223241d6d not found: ID does not exist"
Jul 26 18:36:53.464646 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.464625    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"]
Jul 26 18:36:53.468140 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.468120    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-138d4e-5744c8f76d-s9ffz"]
Jul 26 18:36:53.478209 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.478185    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"]
Jul 26 18:36:53.480601 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:53.480578    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-138d4e-74d98c757d-mqn8d"]
Jul 26 18:36:54.318928 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:54.318895    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fd65d93-e152-475d-b97e-521e341227bc" path="/var/lib/kubelet/pods/3fd65d93-e152-475d-b97e-521e341227bc/volumes"
Jul 26 18:36:54.319311 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:54.319228    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="706051b6-5e60-4965-a013-86f0b1d0700a" path="/var/lib/kubelet/pods/706051b6-5e60-4965-a013-86f0b1d0700a/volumes"
Jul 26 18:36:56.146523 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:56.146467    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"]
Jul 26 18:36:56.148416 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:56.148395    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-138d4e-sts68"]
Jul 26 18:36:56.319294 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:56.319262    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="292aace2-6c74-4755-994e-783b41e628bf" path="/var/lib/kubelet/pods/292aace2-6c74-4755-994e-783b41e628bf/volumes"
Jul 26 18:36:59.326156 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:36:59.326119    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:37:04.185104 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185070    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"]
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185372    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292aace2-6c74-4755-994e-783b41e628bf" containerName="revoke-keys"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185384    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="292aace2-6c74-4755-994e-783b41e628bf" containerName="revoke-keys"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185394    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fd65d93-e152-475d-b97e-521e341227bc" containerName="payload-processing"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185400    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fd65d93-e152-475d-b97e-521e341227bc" containerName="payload-processing"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185415    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706051b6-5e60-4965-a013-86f0b1d0700a" containerName="payload-pre-processing"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185422    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="706051b6-5e60-4965-a013-86f0b1d0700a" containerName="payload-pre-processing"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185488    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="292aace2-6c74-4755-994e-783b41e628bf" containerName="revoke-keys"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185497    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="706051b6-5e60-4965-a013-86f0b1d0700a" containerName="payload-pre-processing"
Jul 26 18:37:04.185512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.185502    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fd65d93-e152-475d-b97e-521e341227bc" containerName="payload-processing"
Jul 26 18:37:04.188373 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.188355    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:04.192280 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.192253    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:37:04.192403 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.192387    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:37:04.199570 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.199541    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"]
Jul 26 18:37:04.210647 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.210619    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a59655-b48d-4af0-a864-ba85795e5b7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f349a2-6clnq\" (UID: \"38a59655-b48d-4af0-a864-ba85795e5b7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:04.311157 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.311122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a59655-b48d-4af0-a864-ba85795e5b7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f349a2-6clnq\" (UID: \"38a59655-b48d-4af0-a864-ba85795e5b7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:04.311756 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.311736    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a59655-b48d-4af0-a864-ba85795e5b7c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-f349a2-6clnq\" (UID: \"38a59655-b48d-4af0-a864-ba85795e5b7c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:04.498619 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.498592    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:04.629863 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:04.629809    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"]
Jul 26 18:37:04.632653 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:37:04.632631    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38a59655_b48d_4af0_a864_ba85795e5b7c.slice/crio-8758c6c1d5695d1b5e0cca6e127a45f8ad77af77fabf7c83c2164eb03f99cf8a WatchSource:0}: Error finding container 8758c6c1d5695d1b5e0cca6e127a45f8ad77af77fabf7c83c2164eb03f99cf8a: Status 404 returned error can't find the container with id 8758c6c1d5695d1b5e0cca6e127a45f8ad77af77fabf7c83c2164eb03f99cf8a
Jul 26 18:37:05.507792 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:05.507761    2565 generic.go:356] "Generic (PLEG): container finished" podID="38a59655-b48d-4af0-a864-ba85795e5b7c" containerID="0393384c79cba5e2f40e16c2c8526d194612b68c04503e2340f8d22fa69e10f9" exitCode=0
Jul 26 18:37:05.508181 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:05.507850    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq" event={"ID":"38a59655-b48d-4af0-a864-ba85795e5b7c","Type":"ContainerDied","Data":"0393384c79cba5e2f40e16c2c8526d194612b68c04503e2340f8d22fa69e10f9"}
Jul 26 18:37:05.508181 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:05.507907    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq" event={"ID":"38a59655-b48d-4af0-a864-ba85795e5b7c","Type":"ContainerStarted","Data":"8758c6c1d5695d1b5e0cca6e127a45f8ad77af77fabf7c83c2164eb03f99cf8a"}
Jul 26 18:37:06.641731 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:06.641688    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:06.731518 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:06.731484    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a59655-b48d-4af0-a864-ba85795e5b7c-maas-api-service-ca\") pod \"38a59655-b48d-4af0-a864-ba85795e5b7c\" (UID: \"38a59655-b48d-4af0-a864-ba85795e5b7c\") "
Jul 26 18:37:06.731880 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:06.731856    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38a59655-b48d-4af0-a864-ba85795e5b7c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38a59655-b48d-4af0-a864-ba85795e5b7c" (UID: "38a59655-b48d-4af0-a864-ba85795e5b7c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:37:06.832055 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:06.831964    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38a59655-b48d-4af0-a864-ba85795e5b7c-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:37:07.519122 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:07.519095    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"
Jul 26 18:37:07.519297 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:07.519092    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq" event={"ID":"38a59655-b48d-4af0-a864-ba85795e5b7c","Type":"ContainerDied","Data":"8758c6c1d5695d1b5e0cca6e127a45f8ad77af77fabf7c83c2164eb03f99cf8a"}
Jul 26 18:37:07.519297 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:07.519226    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8758c6c1d5695d1b5e0cca6e127a45f8ad77af77fabf7c83c2164eb03f99cf8a"
Jul 26 18:37:19.861857 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:19.861821    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"]
Jul 26 18:37:19.862343 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:19.862070    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k" podUID="32a3a315-d552-4f36-b773-0874361e7e90" containerName="payload-processing" containerID="cri-o://f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab" gracePeriod=30
Jul 26 18:37:19.868949 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:19.868915    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"]
Jul 26 18:37:19.869233 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:19.869153    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8" podUID="e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" containerName="payload-pre-processing" containerID="cri-o://27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588" gracePeriod=30
Jul 26 18:37:20.140429 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.140405    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:37:20.150523 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.150501    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:37:20.238237 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.238208    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-plugins-config-volume\") pod \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") "
Jul 26 18:37:20.238237 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.238240    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlql4\" (UniqueName: \"kubernetes.io/projected/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-kube-api-access-dlql4\") pod \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\" (UID: \"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917\") "
Jul 26 18:37:20.238499 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.238268    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a3a315-d552-4f36-b773-0874361e7e90-plugins-config-volume\") pod \"32a3a315-d552-4f36-b773-0874361e7e90\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") "
Jul 26 18:37:20.238499 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.238304    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lx8l\" (UniqueName: \"kubernetes.io/projected/32a3a315-d552-4f36-b773-0874361e7e90-kube-api-access-8lx8l\") pod \"32a3a315-d552-4f36-b773-0874361e7e90\" (UID: \"32a3a315-d552-4f36-b773-0874361e7e90\") "
Jul 26 18:37:20.238642 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.238606    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a3a315-d552-4f36-b773-0874361e7e90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32a3a315-d552-4f36-b773-0874361e7e90" (UID: "32a3a315-d552-4f36-b773-0874361e7e90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:37:20.238756 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.238732    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" (UID: "e51ff061-9ce9-46f6-ba6c-c6c95bdd8917"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:37:20.240400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.240379    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a3a315-d552-4f36-b773-0874361e7e90-kube-api-access-8lx8l" (OuterVolumeSpecName: "kube-api-access-8lx8l") pod "32a3a315-d552-4f36-b773-0874361e7e90" (UID: "32a3a315-d552-4f36-b773-0874361e7e90"). InnerVolumeSpecName "kube-api-access-8lx8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:37:20.240467 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.240429    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-kube-api-access-dlql4" (OuterVolumeSpecName: "kube-api-access-dlql4") pod "e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" (UID: "e51ff061-9ce9-46f6-ba6c-c6c95bdd8917"). InnerVolumeSpecName "kube-api-access-dlql4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:37:20.339816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.339785    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:37:20.339816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.339813    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlql4\" (UniqueName: \"kubernetes.io/projected/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917-kube-api-access-dlql4\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:37:20.339816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.339822    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a3a315-d552-4f36-b773-0874361e7e90-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:37:20.340034 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.339831    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lx8l\" (UniqueName: \"kubernetes.io/projected/32a3a315-d552-4f36-b773-0874361e7e90-kube-api-access-8lx8l\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:37:20.587402 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.587364    2565 generic.go:356] "Generic (PLEG): container finished" podID="32a3a315-d552-4f36-b773-0874361e7e90" containerID="f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab" exitCode=0
Jul 26 18:37:20.587569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.587411    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"
Jul 26 18:37:20.587569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.587455    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k" event={"ID":"32a3a315-d552-4f36-b773-0874361e7e90","Type":"ContainerDied","Data":"f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab"}
Jul 26 18:37:20.587569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.587508    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k" event={"ID":"32a3a315-d552-4f36-b773-0874361e7e90","Type":"ContainerDied","Data":"47c7bd189951d568a579bff0c8f9f527ce642bdd82e88055e47399920b2083f9"}
Jul 26 18:37:20.587569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.587539    2565 scope.go:117] "RemoveContainer" containerID="f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab"
Jul 26 18:37:20.588945 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.588919    2565 generic.go:356] "Generic (PLEG): container finished" podID="e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" containerID="27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588" exitCode=0
Jul 26 18:37:20.589050 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.588967    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"
Jul 26 18:37:20.589050 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.588994    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8" event={"ID":"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917","Type":"ContainerDied","Data":"27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588"}
Jul 26 18:37:20.589050 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.589046    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8" event={"ID":"e51ff061-9ce9-46f6-ba6c-c6c95bdd8917","Type":"ContainerDied","Data":"b841a79034a995ce820a25bb810782281e7f4cf723040a3b316de364e6e43e4d"}
Jul 26 18:37:20.596617 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.596600    2565 scope.go:117] "RemoveContainer" containerID="f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab"
Jul 26 18:37:20.596890 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:37:20.596874    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab\": container with ID starting with f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab not found: ID does not exist" containerID="f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab"
Jul 26 18:37:20.596941 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.596900    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab"} err="failed to get container status \"f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab\": rpc error: code = NotFound desc = could not find container \"f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab\": container with ID starting with f62033b01a838e26bf17f27cce932c5b0c3b6507505490beb82dedc8a14d85ab not found: ID does not exist"
Jul 26 18:37:20.596941 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.596919    2565 scope.go:117] "RemoveContainer" containerID="27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588"
Jul 26 18:37:20.604679 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.604661    2565 scope.go:117] "RemoveContainer" containerID="27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588"
Jul 26 18:37:20.605017 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:37:20.604991    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588\": container with ID starting with 27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588 not found: ID does not exist" containerID="27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588"
Jul 26 18:37:20.605293 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.605024    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588"} err="failed to get container status \"27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588\": rpc error: code = NotFound desc = could not find container \"27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588\": container with ID starting with 27c4f656c9b2bda7392a638876aec738fbde9bfe90f5ec1ce4ab6ace32fd4588 not found: ID does not exist"
Jul 26 18:37:20.606903 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.606885    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"]
Jul 26 18:37:20.614021 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.613999    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-f349a2-6c646777b-4cd5k"]
Jul 26 18:37:20.625743 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.625719    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"]
Jul 26 18:37:20.631349 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:20.631327    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f349a2-54c579b984-5xww8"]
Jul 26 18:37:22.318535 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:22.318506    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32a3a315-d552-4f36-b773-0874361e7e90" path="/var/lib/kubelet/pods/32a3a315-d552-4f36-b773-0874361e7e90/volumes"
Jul 26 18:37:22.318931 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:22.318852    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" path="/var/lib/kubelet/pods/e51ff061-9ce9-46f6-ba6c-c6c95bdd8917/volumes"
Jul 26 18:37:24.322964 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:24.322933    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"]
Jul 26 18:37:24.326241 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:24.326219    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-f349a2-6clnq"]
Jul 26 18:37:24.764438 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:24.761511    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:37:26.318881 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:26.318848    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38a59655-b48d-4af0-a864-ba85795e5b7c" path="/var/lib/kubelet/pods/38a59655-b48d-4af0-a864-ba85795e5b7c/volumes"
Jul 26 18:37:36.024100 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024064    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"]
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024346    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a3a315-d552-4f36-b773-0874361e7e90" containerName="payload-processing"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024357    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a3a315-d552-4f36-b773-0874361e7e90" containerName="payload-processing"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024380    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" containerName="payload-pre-processing"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024386    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" containerName="payload-pre-processing"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024393    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38a59655-b48d-4af0-a864-ba85795e5b7c" containerName="revoke-keys"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024399    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="38a59655-b48d-4af0-a864-ba85795e5b7c" containerName="revoke-keys"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024451    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a3a315-d552-4f36-b773-0874361e7e90" containerName="payload-processing"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024457    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e51ff061-9ce9-46f6-ba6c-c6c95bdd8917" containerName="payload-pre-processing"
Jul 26 18:37:36.024480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.024463    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="38a59655-b48d-4af0-a864-ba85795e5b7c" containerName="revoke-keys"
Jul 26 18:37:36.027243 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.027226    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.031264 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.031240    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-37e93d-dockercfg-nwt6k\""
Jul 26 18:37:36.031404 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.031240    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-37e93d\""
Jul 26 18:37:36.041306 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.041285    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"]
Jul 26 18:37:36.058350 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.058324    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba4853c-f927-44e9-a29b-c684df532f6b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.058455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.058384    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z947r\" (UniqueName: \"kubernetes.io/projected/6ba4853c-f927-44e9-a29b-c684df532f6b-kube-api-access-z947r\") pod \"payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.077406 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.077374    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"]
Jul 26 18:37:36.080658 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.080643    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.090298 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.090254    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"]
Jul 26 18:37:36.159841 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.159808    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8vfl\" (UniqueName: \"kubernetes.io/projected/11857142-854f-4cbb-a8fe-bbba3707ee3a-kube-api-access-w8vfl\") pod \"payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.160012 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.159848    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba4853c-f927-44e9-a29b-c684df532f6b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.160012 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.159932    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11857142-854f-4cbb-a8fe-bbba3707ee3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.160012 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.159979    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z947r\" (UniqueName: \"kubernetes.io/projected/6ba4853c-f927-44e9-a29b-c684df532f6b-kube-api-access-z947r\") pod \"payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.160455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.160439    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba4853c-f927-44e9-a29b-c684df532f6b-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.169408 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.169382    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z947r\" (UniqueName: \"kubernetes.io/projected/6ba4853c-f927-44e9-a29b-c684df532f6b-kube-api-access-z947r\") pod \"payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") " pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.261156 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.261123    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8vfl\" (UniqueName: \"kubernetes.io/projected/11857142-854f-4cbb-a8fe-bbba3707ee3a-kube-api-access-w8vfl\") pod \"payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.261341 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.261186    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11857142-854f-4cbb-a8fe-bbba3707ee3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.261874 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.261856    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11857142-854f-4cbb-a8fe-bbba3707ee3a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.269965 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.269938    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8vfl\" (UniqueName: \"kubernetes.io/projected/11857142-854f-4cbb-a8fe-bbba3707ee3a-kube-api-access-w8vfl\") pod \"payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.336735 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.336649    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.390974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.390919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.479960 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.479676    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"]
Jul 26 18:37:36.481189 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:37:36.481158    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba4853c_f927_44e9_a29b_c684df532f6b.slice/crio-fdc0e21fab08cabcb67aad6fd070bf0c106198f0b00f1fd902a23676fb7b5873 WatchSource:0}: Error finding container fdc0e21fab08cabcb67aad6fd070bf0c106198f0b00f1fd902a23676fb7b5873: Status 404 returned error can't find the container with id fdc0e21fab08cabcb67aad6fd070bf0c106198f0b00f1fd902a23676fb7b5873
Jul 26 18:37:36.535748 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.535719    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"]
Jul 26 18:37:36.537208 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:37:36.537182    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11857142_854f_4cbb_a8fe_bbba3707ee3a.slice/crio-b77986e8f687c13927072fd98b0b18434eda1b2e9c1ed5bc6b06997d057e5e2b WatchSource:0}: Error finding container b77986e8f687c13927072fd98b0b18434eda1b2e9c1ed5bc6b06997d057e5e2b: Status 404 returned error can't find the container with id b77986e8f687c13927072fd98b0b18434eda1b2e9c1ed5bc6b06997d057e5e2b
Jul 26 18:37:36.676953 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.676914    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988" event={"ID":"11857142-854f-4cbb-a8fe-bbba3707ee3a","Type":"ContainerStarted","Data":"980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b"}
Jul 26 18:37:36.677124 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.677029    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988" event={"ID":"11857142-854f-4cbb-a8fe-bbba3707ee3a","Type":"ContainerStarted","Data":"b77986e8f687c13927072fd98b0b18434eda1b2e9c1ed5bc6b06997d057e5e2b"}
Jul 26 18:37:36.677217 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.677196    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:36.678765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.678736    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj" event={"ID":"6ba4853c-f927-44e9-a29b-c684df532f6b","Type":"ContainerStarted","Data":"f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf"}
Jul 26 18:37:36.678871 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.678785    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj" event={"ID":"6ba4853c-f927-44e9-a29b-c684df532f6b","Type":"ContainerStarted","Data":"fdc0e21fab08cabcb67aad6fd070bf0c106198f0b00f1fd902a23676fb7b5873"}
Jul 26 18:37:36.678871 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.678813    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:36.698971 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.698627    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988" podStartSLOduration=0.698609347 podStartE2EDuration="698.609347ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:37:36.698561384 +0000 UTC m=+2410.888591139" watchObservedRunningTime="2026-07-26 18:37:36.698609347 +0000 UTC m=+2410.888639107"
Jul 26 18:37:36.727228 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:36.727179    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj" podStartSLOduration=0.727164341 podStartE2EDuration="727.164341ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:37:36.725402276 +0000 UTC m=+2410.915432030" watchObservedRunningTime="2026-07-26 18:37:36.727164341 +0000 UTC m=+2410.917194094"
Jul 26 18:37:42.818112 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:42.818076    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:37:44.249816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.249779    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"]
Jul 26 18:37:44.253464 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.253443    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.256864 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.256840    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-d112e4\""
Jul 26 18:37:44.257181 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.257165    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-d112e4-dockercfg-58mkq\""
Jul 26 18:37:44.270154 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.270129    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"]
Jul 26 18:37:44.329478 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.329449    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25953ca-740d-4ca8-adb6-c2c08fcb7401-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.329650 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.329537    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkzw9\" (UniqueName: \"kubernetes.io/projected/e25953ca-740d-4ca8-adb6-c2c08fcb7401-kube-api-access-gkzw9\") pod \"payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.340944 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.340913    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"]
Jul 26 18:37:44.344218 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.344196    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.354549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.354522    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"]
Jul 26 18:37:44.430215 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.430175    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.430417 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.430232    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkzw9\" (UniqueName: \"kubernetes.io/projected/e25953ca-740d-4ca8-adb6-c2c08fcb7401-kube-api-access-gkzw9\") pod \"payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.430417 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.430262    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfjrh\" (UniqueName: \"kubernetes.io/projected/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-kube-api-access-jfjrh\") pod \"payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.430417 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.430295    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25953ca-740d-4ca8-adb6-c2c08fcb7401-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.431315 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.431293    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25953ca-740d-4ca8-adb6-c2c08fcb7401-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.439204 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.439181    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkzw9\" (UniqueName: \"kubernetes.io/projected/e25953ca-740d-4ca8-adb6-c2c08fcb7401-kube-api-access-gkzw9\") pod \"payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") " pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.531519 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.531417    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.531519 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.531468    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfjrh\" (UniqueName: \"kubernetes.io/projected/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-kube-api-access-jfjrh\") pod \"payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.532231 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.532204    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.540502 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.540478    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfjrh\" (UniqueName: \"kubernetes.io/projected/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-kube-api-access-jfjrh\") pod \"payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.563055 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.563022    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:44.655084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.655050    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:44.702383 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.702357    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"]
Jul 26 18:37:44.704063 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:37:44.704035    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25953ca_740d_4ca8_adb6_c2c08fcb7401.slice/crio-d4c85f5e42628b18981d447f923119af86a1e563b72901734ac8b5c318651785 WatchSource:0}: Error finding container d4c85f5e42628b18981d447f923119af86a1e563b72901734ac8b5c318651785: Status 404 returned error can't find the container with id d4c85f5e42628b18981d447f923119af86a1e563b72901734ac8b5c318651785
Jul 26 18:37:44.725231 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.725199    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5" event={"ID":"e25953ca-740d-4ca8-adb6-c2c08fcb7401","Type":"ContainerStarted","Data":"d4c85f5e42628b18981d447f923119af86a1e563b72901734ac8b5c318651785"}
Jul 26 18:37:44.798575 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:44.798551    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"]
Jul 26 18:37:44.800867 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:37:44.800834    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80772fc_f4fd_41b8_bb0d_6ba1012ae42f.slice/crio-d918def744cb5fda1fb8f8043381e60cf21304dcc152539af5aff58340e89767 WatchSource:0}: Error finding container d918def744cb5fda1fb8f8043381e60cf21304dcc152539af5aff58340e89767: Status 404 returned error can't find the container with id d918def744cb5fda1fb8f8043381e60cf21304dcc152539af5aff58340e89767
Jul 26 18:37:45.731987 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.731948    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2" event={"ID":"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f","Type":"ContainerStarted","Data":"562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59"}
Jul 26 18:37:45.732469 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.732121    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:45.732469 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.732173    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2" event={"ID":"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f","Type":"ContainerStarted","Data":"d918def744cb5fda1fb8f8043381e60cf21304dcc152539af5aff58340e89767"}
Jul 26 18:37:45.733424 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.733405    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5" event={"ID":"e25953ca-740d-4ca8-adb6-c2c08fcb7401","Type":"ContainerStarted","Data":"bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e"}
Jul 26 18:37:45.733543 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.733526    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:45.759856 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.759810    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2" podStartSLOduration=1.759795471 podStartE2EDuration="1.759795471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:37:45.757201681 +0000 UTC m=+2419.947231435" watchObservedRunningTime="2026-07-26 18:37:45.759795471 +0000 UTC m=+2419.949825202"
Jul 26 18:37:45.785554 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:45.785492    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5" podStartSLOduration=1.785471177 podStartE2EDuration="1.785471177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:37:45.7827555 +0000 UTC m=+2419.972785254" watchObservedRunningTime="2026-07-26 18:37:45.785471177 +0000 UTC m=+2419.975500932"
Jul 26 18:37:47.686742 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:47.686687    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:37:47.687131 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:47.686831    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:37:56.033360 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.033325    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"]
Jul 26 18:37:56.037284 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.037263    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.046946 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.046924    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"openshift-service-ca.crt\""
Jul 26 18:37:56.047083 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.047020    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"kube-root-ca.crt\""
Jul 26 18:37:56.048385 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.048359    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"auth-test-model-37e93d-kserve-self-signed-certs\""
Jul 26 18:37:56.048502 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.048396    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"default-dockercfg-678bm\""
Jul 26 18:37:56.050807 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.050784    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"]
Jul 26 18:37:56.135380 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.135347    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-model-cache\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.135380 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.135389    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd7jj\" (UniqueName: \"kubernetes.io/projected/59d8834b-9c2f-4613-9ea0-43b3214e1509-kube-api-access-gd7jj\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.135583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.135412    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8834b-9c2f-4613-9ea0-43b3214e1509-tls-certs\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.135583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.135437    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-tmp-dir\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.135583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.135498    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-dshm\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.135583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.135524    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-home\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236279 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236245    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-home\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236302    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-model-cache\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236334    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd7jj\" (UniqueName: \"kubernetes.io/projected/59d8834b-9c2f-4613-9ea0-43b3214e1509-kube-api-access-gd7jj\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8834b-9c2f-4613-9ea0-43b3214e1509-tls-certs\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236403    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-tmp-dir\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236461    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-dshm\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236786 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236764    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-home\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236846 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236805    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-model-cache\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.236893 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.236873    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-tmp-dir\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.238730 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.238673    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-dshm\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.238941 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.238922    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8834b-9c2f-4613-9ea0-43b3214e1509-tls-certs\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.244965 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.244946    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd7jj\" (UniqueName: \"kubernetes.io/projected/59d8834b-9c2f-4613-9ea0-43b3214e1509-kube-api-access-gd7jj\") pod \"auth-test-model-37e93d-kserve-54d9485b8c-vzggr\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") " pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.346422 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.346337    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:37:56.483887 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.483858    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"]
Jul 26 18:37:56.485042 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:37:56.485018    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59d8834b_9c2f_4613_9ea0_43b3214e1509.slice/crio-2bb449853a6899b507ae92233f727ba61268ef4fe72b6d44f8c22ef1da4a29e4 WatchSource:0}: Error finding container 2bb449853a6899b507ae92233f727ba61268ef4fe72b6d44f8c22ef1da4a29e4: Status 404 returned error can't find the container with id 2bb449853a6899b507ae92233f727ba61268ef4fe72b6d44f8c22ef1da4a29e4
Jul 26 18:37:56.740713 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.740669    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:37:56.741124 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.741105    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:37:56.794075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.794019    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr" event={"ID":"59d8834b-9c2f-4613-9ea0-43b3214e1509","Type":"ContainerStarted","Data":"3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a"}
Jul 26 18:37:56.794075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.794074    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr" event={"ID":"59d8834b-9c2f-4613-9ea0-43b3214e1509","Type":"ContainerStarted","Data":"2bb449853a6899b507ae92233f727ba61268ef4fe72b6d44f8c22ef1da4a29e4"}
Jul 26 18:37:56.813494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:56.813443    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr" podStartSLOduration=0.813428102 podStartE2EDuration="813.428102ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:37:56.811962083 +0000 UTC m=+2431.001991838" watchObservedRunningTime="2026-07-26 18:37:56.813428102 +0000 UTC m=+2431.003457856"
Jul 26 18:37:57.424511 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:37:57.424472    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:38:06.347371 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:06.347335    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:38:06.347870 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:06.347386    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:38:06.359765 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:06.359743    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:38:06.858600 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:06.858574    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:38:08.929010 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:08.928976    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:38:21.628339 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:21.628305    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:38:26.542943 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:26.542910    2565 scope.go:117] "RemoveContainer" containerID="7a61aa86ed5b867ccf61bb637de2fc244f36577880a9d90e8b1d47c910cdc8a0"
Jul 26 18:38:49.031090 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:38:49.031048    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:39:09.444115 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:39:09.444078    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:39:23.819038 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:39:23.818960    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:39:35.637244 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:39:35.637209    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:39:58.228563 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:39:58.228522    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:40:15.813593 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:40:15.813551    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:40:43.023552 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:40:43.023497    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:41:01.231895 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:41:01.231817    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:41:23.619251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:41:23.619217    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:41:41.618662 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:41:41.618628    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:42:02.826457 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:42:02.826420    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:42:21.620005 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:42:21.619925    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:42:26.586885 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:42:26.586855    2565 scope.go:117] "RemoveContainer" containerID="24fd9de890a6b1dc02034e4eb6511d89319b5d7ebeef3b70224107da62298225"
Jul 26 18:42:26.594756 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:42:26.594737    2565 scope.go:117] "RemoveContainer" containerID="27a59db29039671dc7a6bc9ae506e667da60fdede00055ddfea42e7b4191a118"
Jul 26 18:42:44.807832 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:42:44.807794    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:42:48.956994 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:42:48.956958    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"]
Jul 26 18:43:01.810434 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:01.810391    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:43:03.975642 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:03.975580    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr" podUID="59d8834b-9c2f-4613-9ea0-43b3214e1509" containerName="main" containerID="cri-o://3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a" gracePeriod=45
Jul 26 18:43:04.228329 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.228262    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:43:04.246573 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.246529    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-dshm\") pod \"59d8834b-9c2f-4613-9ea0-43b3214e1509\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") "
Jul 26 18:43:04.246755 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.246620    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-tmp-dir\") pod \"59d8834b-9c2f-4613-9ea0-43b3214e1509\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") "
Jul 26 18:43:04.246755 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.246658    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-home\") pod \"59d8834b-9c2f-4613-9ea0-43b3214e1509\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") "
Jul 26 18:43:04.246755 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.246747    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd7jj\" (UniqueName: \"kubernetes.io/projected/59d8834b-9c2f-4613-9ea0-43b3214e1509-kube-api-access-gd7jj\") pod \"59d8834b-9c2f-4613-9ea0-43b3214e1509\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") "
Jul 26 18:43:04.246955 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.246782    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-model-cache\") pod \"59d8834b-9c2f-4613-9ea0-43b3214e1509\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") "
Jul 26 18:43:04.246955 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.246810    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8834b-9c2f-4613-9ea0-43b3214e1509-tls-certs\") pod \"59d8834b-9c2f-4613-9ea0-43b3214e1509\" (UID: \"59d8834b-9c2f-4613-9ea0-43b3214e1509\") "
Jul 26 18:43:04.247113 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.247084    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-home" (OuterVolumeSpecName: "home") pod "59d8834b-9c2f-4613-9ea0-43b3214e1509" (UID: "59d8834b-9c2f-4613-9ea0-43b3214e1509"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:43:04.247246 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.247218    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59d8834b-9c2f-4613-9ea0-43b3214e1509" (UID: "59d8834b-9c2f-4613-9ea0-43b3214e1509"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:43:04.247388 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.247365    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-model-cache" (OuterVolumeSpecName: "model-cache") pod "59d8834b-9c2f-4613-9ea0-43b3214e1509" (UID: "59d8834b-9c2f-4613-9ea0-43b3214e1509"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:43:04.249724 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.249681    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59d8834b-9c2f-4613-9ea0-43b3214e1509-kube-api-access-gd7jj" (OuterVolumeSpecName: "kube-api-access-gd7jj") pod "59d8834b-9c2f-4613-9ea0-43b3214e1509" (UID: "59d8834b-9c2f-4613-9ea0-43b3214e1509"). InnerVolumeSpecName "kube-api-access-gd7jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:43:04.249817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.249741    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-dshm" (OuterVolumeSpecName: "dshm") pod "59d8834b-9c2f-4613-9ea0-43b3214e1509" (UID: "59d8834b-9c2f-4613-9ea0-43b3214e1509"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:43:04.249817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.249778    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59d8834b-9c2f-4613-9ea0-43b3214e1509-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59d8834b-9c2f-4613-9ea0-43b3214e1509" (UID: "59d8834b-9c2f-4613-9ea0-43b3214e1509"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:43:04.348111 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.348076    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:43:04.348111 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.348104    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:43:04.348111 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.348115    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd7jj\" (UniqueName: \"kubernetes.io/projected/59d8834b-9c2f-4613-9ea0-43b3214e1509-kube-api-access-gd7jj\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:43:04.348324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.348126    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:43:04.348324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.348135    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59d8834b-9c2f-4613-9ea0-43b3214e1509-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:43:04.348324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.348142    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59d8834b-9c2f-4613-9ea0-43b3214e1509-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:43:04.363601 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.363572    2565 generic.go:356] "Generic (PLEG): container finished" podID="59d8834b-9c2f-4613-9ea0-43b3214e1509" containerID="3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a" exitCode=0
Jul 26 18:43:04.363752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.363613    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"
Jul 26 18:43:04.363752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.363656    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr" event={"ID":"59d8834b-9c2f-4613-9ea0-43b3214e1509","Type":"ContainerDied","Data":"3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a"}
Jul 26 18:43:04.363752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.363709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr" event={"ID":"59d8834b-9c2f-4613-9ea0-43b3214e1509","Type":"ContainerDied","Data":"2bb449853a6899b507ae92233f727ba61268ef4fe72b6d44f8c22ef1da4a29e4"}
Jul 26 18:43:04.363752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.363728    2565 scope.go:117] "RemoveContainer" containerID="3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a"
Jul 26 18:43:04.372442 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.372420    2565 scope.go:117] "RemoveContainer" containerID="3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a"
Jul 26 18:43:04.372725 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:43:04.372685    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a\": container with ID starting with 3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a not found: ID does not exist" containerID="3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a"
Jul 26 18:43:04.372797 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.372728    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a"} err="failed to get container status \"3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a\": rpc error: code = NotFound desc = could not find container \"3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a\": container with ID starting with 3dd09aac4a49b3c023c32fa5460d9215bf22c7606b4a97e1b7a2bba1fb05ce6a not found: ID does not exist"
Jul 26 18:43:04.384378 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.384348    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"]
Jul 26 18:43:04.387576 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:04.387555    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/auth-test-model-37e93d-kserve-54d9485b8c-vzggr"]
Jul 26 18:43:06.318823 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:06.318790    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59d8834b-9c2f-4613-9ea0-43b3214e1509" path="/var/lib/kubelet/pods/59d8834b-9c2f-4613-9ea0-43b3214e1509/volumes"
Jul 26 18:43:09.535172 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.535128    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"]
Jul 26 18:43:09.535797 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.535596    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59d8834b-9c2f-4613-9ea0-43b3214e1509" containerName="main"
Jul 26 18:43:09.535797 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.535614    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="59d8834b-9c2f-4613-9ea0-43b3214e1509" containerName="main"
Jul 26 18:43:09.535797 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.535725    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="59d8834b-9c2f-4613-9ea0-43b3214e1509" containerName="main"
Jul 26 18:43:09.538694 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.538674    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.542396 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.542374    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"kube-root-ca.crt\""
Jul 26 18:43:09.542514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.542449    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"openshift-service-ca.crt\""
Jul 26 18:43:09.542514 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.542493    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"default-dockercfg-678bm\""
Jul 26 18:43:09.542621 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.542556    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"sub-test-model-37e93d-kserve-self-signed-certs\""
Jul 26 18:43:09.547610 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.547587    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"]
Jul 26 18:43:09.588534 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.588497    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-dshm\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.588749 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.588543    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tls-certs\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.588749 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.588566    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tmp-dir\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.588749 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.588606    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-home\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.588749 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.588684    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9xck\" (UniqueName: \"kubernetes.io/projected/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-kube-api-access-p9xck\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.588937 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.588757    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-model-cache\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689143 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689108    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tmp-dir\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689350 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689149    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-home\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689350 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689213    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9xck\" (UniqueName: \"kubernetes.io/projected/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-kube-api-access-p9xck\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689350 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689257    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-model-cache\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689525 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689428    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-dshm\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689525 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689490    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tls-certs\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689525 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689495    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tmp-dir\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689538    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-home\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.689690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.689600    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-model-cache\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.691632 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.691611    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-dshm\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.691788 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.691772    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tls-certs\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.702978 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.702953    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9xck\" (UniqueName: \"kubernetes.io/projected/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-kube-api-access-p9xck\") pod \"sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") " pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.847851 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.847753    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:09.980863 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:09.980802    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"]
Jul 26 18:43:09.982944 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:43:09.982906    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80d8e5b1_2a62_4bbb_98e7_34c719f08bf7.slice/crio-428848f62ee6965f3f3d03ee59bb728f48b1f28b72184c3325426669d7eb0995 WatchSource:0}: Error finding container 428848f62ee6965f3f3d03ee59bb728f48b1f28b72184c3325426669d7eb0995: Status 404 returned error can't find the container with id 428848f62ee6965f3f3d03ee59bb728f48b1f28b72184c3325426669d7eb0995
Jul 26 18:43:10.396236 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:10.396191    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm" event={"ID":"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7","Type":"ContainerStarted","Data":"bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d"}
Jul 26 18:43:10.396236 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:10.396242    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm" event={"ID":"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7","Type":"ContainerStarted","Data":"428848f62ee6965f3f3d03ee59bb728f48b1f28b72184c3325426669d7eb0995"}
Jul 26 18:43:10.415690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:10.415641    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm" podStartSLOduration=1.415625827 podStartE2EDuration="1.415625827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:10.414497731 +0000 UTC m=+2744.604527485" watchObservedRunningTime="2026-07-26 18:43:10.415625827 +0000 UTC m=+2744.605655581"
Jul 26 18:43:14.114793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:14.114755    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:43:19.848039 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:19.847997    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:19.848039 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:19.848050    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:19.860715 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:19.860673    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:20.461899 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:20.461855    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:43:26.616167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:26.616137    2565 scope.go:117] "RemoveContainer" containerID="0393384c79cba5e2f40e16c2c8526d194612b68c04503e2340f8d22fa69e10f9"
Jul 26 18:43:26.624162 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:26.624142    2565 scope.go:117] "RemoveContainer" containerID="ef6a38b3deeac646c884a053d7b83c352bbea88aefa98d552dccfbae4d626cec"
Jul 26 18:43:28.041650 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.041612    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"]
Jul 26 18:43:28.046493 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.046474    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.049071 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.049048    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"kube-root-ca.crt\""
Jul 26 18:43:28.049186 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.049114    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"openshift-service-ca.crt\""
Jul 26 18:43:28.050121 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.050106    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"default-dockercfg-5h9xv\""
Jul 26 18:43:28.050180 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.050172    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"sub-test-model-d112e4-kserve-self-signed-certs\""
Jul 26 18:43:28.055863 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.055841    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"]
Jul 26 18:43:28.153591 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.153554    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-dshm\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.153789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.153601    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-tmp-dir\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.153789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.153656    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d05114-2b2a-4677-925e-d0a9bb16032e-tls-certs\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.153789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.153729    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-home\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.153789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.153773    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-model-cache\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.153995 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.153868    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w78ck\" (UniqueName: \"kubernetes.io/projected/53d05114-2b2a-4677-925e-d0a9bb16032e-kube-api-access-w78ck\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254261 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254225    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-model-cache\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254285    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w78ck\" (UniqueName: \"kubernetes.io/projected/53d05114-2b2a-4677-925e-d0a9bb16032e-kube-api-access-w78ck\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254308    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-dshm\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254329    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-tmp-dir\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254349    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d05114-2b2a-4677-925e-d0a9bb16032e-tls-certs\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254458 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254370    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-home\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254785 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254758    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-model-cache\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254845 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254793    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-tmp-dir\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.254845 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.254835    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-home\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.256824 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.256787    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-dshm\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.257056 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.257037    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d05114-2b2a-4677-925e-d0a9bb16032e-tls-certs\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.263942 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.263905    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w78ck\" (UniqueName: \"kubernetes.io/projected/53d05114-2b2a-4677-925e-d0a9bb16032e-kube-api-access-w78ck\") pod \"sub-test-model-d112e4-kserve-85cb5f8685-9c7rt\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") " pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.356385 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.356285    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:28.492663 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:28.492637    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"]
Jul 26 18:43:28.494420 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:43:28.494397    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53d05114_2b2a_4677_925e_d0a9bb16032e.slice/crio-608a3d937e992570fb7d8e8acb8c50d1c232c3dcbafa9f3b3f2008363f31e436 WatchSource:0}: Error finding container 608a3d937e992570fb7d8e8acb8c50d1c232c3dcbafa9f3b3f2008363f31e436: Status 404 returned error can't find the container with id 608a3d937e992570fb7d8e8acb8c50d1c232c3dcbafa9f3b3f2008363f31e436
Jul 26 18:43:29.494452 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:29.494417    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt" event={"ID":"53d05114-2b2a-4677-925e-d0a9bb16032e","Type":"ContainerStarted","Data":"072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae"}
Jul 26 18:43:29.494452 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:29.494456    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt" event={"ID":"53d05114-2b2a-4677-925e-d0a9bb16032e","Type":"ContainerStarted","Data":"608a3d937e992570fb7d8e8acb8c50d1c232c3dcbafa9f3b3f2008363f31e436"}
Jul 26 18:43:29.515379 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:29.515328    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt" podStartSLOduration=1.515314199 podStartE2EDuration="1.515314199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:29.512517736 +0000 UTC m=+2763.702547482" watchObservedRunningTime="2026-07-26 18:43:29.515314199 +0000 UTC m=+2763.705343952"
Jul 26 18:43:31.817953 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:31.817920    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:43:38.356474 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:38.356418    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:38.356969 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:38.356563    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:38.369437 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:38.369408    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:43:38.548231 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:43:38.548195    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:44:03.217638 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:03.217605    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:44:20.505920 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:20.505883    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:44:31.074952 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:31.074919    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"]
Jul 26 18:44:31.920902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:31.920871    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"]
Jul 26 18:44:34.830364 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:34.830330    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:44:46.094148 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.094090    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm" podUID="80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" containerName="main" containerID="cri-o://bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d" gracePeriod=45
Jul 26 18:44:46.347990 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.347932    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:44:46.499450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499419    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tls-certs\") pod \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") "
Jul 26 18:44:46.499450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499457    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9xck\" (UniqueName: \"kubernetes.io/projected/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-kube-api-access-p9xck\") pod \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") "
Jul 26 18:44:46.499673 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499548    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-home\") pod \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") "
Jul 26 18:44:46.499673 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499661    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-model-cache\") pod \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") "
Jul 26 18:44:46.499816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499739    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-dshm\") pod \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") "
Jul 26 18:44:46.499816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499770    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tmp-dir\") pod \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\" (UID: \"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7\") "
Jul 26 18:44:46.499816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499805    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-home" (OuterVolumeSpecName: "home") pod "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" (UID: "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:46.499974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.499935    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-model-cache" (OuterVolumeSpecName: "model-cache") pod "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" (UID: "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:46.500095 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.500072    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:46.500095 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.500091    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:46.500194 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.500103    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" (UID: "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:46.501654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.501620    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-dshm" (OuterVolumeSpecName: "dshm") pod "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" (UID: "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:46.501789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.501685    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" (UID: "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:44:46.501789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.501691    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-kube-api-access-p9xck" (OuterVolumeSpecName: "kube-api-access-p9xck") pod "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" (UID: "80d8e5b1-2a62-4bbb-98e7-34c719f08bf7"). InnerVolumeSpecName "kube-api-access-p9xck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:44:46.600959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.600878    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:46.600959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.600905    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:46.600959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.600914    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:46.600959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.600923    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9xck\" (UniqueName: \"kubernetes.io/projected/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7-kube-api-access-p9xck\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:46.881621 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.881539    2565 generic.go:356] "Generic (PLEG): container finished" podID="80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" containerID="bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d" exitCode=0
Jul 26 18:44:46.881621 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.881579    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"
Jul 26 18:44:46.881831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.881620    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm" event={"ID":"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7","Type":"ContainerDied","Data":"bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d"}
Jul 26 18:44:46.881831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.881667    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm" event={"ID":"80d8e5b1-2a62-4bbb-98e7-34c719f08bf7","Type":"ContainerDied","Data":"428848f62ee6965f3f3d03ee59bb728f48b1f28b72184c3325426669d7eb0995"}
Jul 26 18:44:46.881831 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.881685    2565 scope.go:117] "RemoveContainer" containerID="bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d"
Jul 26 18:44:46.890971 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.890954    2565 scope.go:117] "RemoveContainer" containerID="bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d"
Jul 26 18:44:46.891204 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:44:46.891187    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d\": container with ID starting with bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d not found: ID does not exist" containerID="bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d"
Jul 26 18:44:46.891251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.891212    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d"} err="failed to get container status \"bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d\": rpc error: code = NotFound desc = could not find container \"bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d\": container with ID starting with bc4dae3685157eeaab78cd8e8ece894e95577a37c335d3c2ed5fcd41d727633d not found: ID does not exist"
Jul 26 18:44:46.902919 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.902894    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"]
Jul 26 18:44:46.908933 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.908913    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/sub-test-model-37e93d-kserve-5f6d8fb6d9-qfrpm"]
Jul 26 18:44:46.939690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:46.939654    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt" podUID="53d05114-2b2a-4677-925e-d0a9bb16032e" containerName="main" containerID="cri-o://072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae" gracePeriod=45
Jul 26 18:44:47.192874 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.192849    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:44:47.306684 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306645    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-dshm\") pod \"53d05114-2b2a-4677-925e-d0a9bb16032e\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") "
Jul 26 18:44:47.306905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306719    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d05114-2b2a-4677-925e-d0a9bb16032e-tls-certs\") pod \"53d05114-2b2a-4677-925e-d0a9bb16032e\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") "
Jul 26 18:44:47.306905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306746    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-tmp-dir\") pod \"53d05114-2b2a-4677-925e-d0a9bb16032e\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") "
Jul 26 18:44:47.306905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306790    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-home\") pod \"53d05114-2b2a-4677-925e-d0a9bb16032e\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") "
Jul 26 18:44:47.306905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306839    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w78ck\" (UniqueName: \"kubernetes.io/projected/53d05114-2b2a-4677-925e-d0a9bb16032e-kube-api-access-w78ck\") pod \"53d05114-2b2a-4677-925e-d0a9bb16032e\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") "
Jul 26 18:44:47.306905 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306885    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-model-cache\") pod \"53d05114-2b2a-4677-925e-d0a9bb16032e\" (UID: \"53d05114-2b2a-4677-925e-d0a9bb16032e\") "
Jul 26 18:44:47.307212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.306996    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "53d05114-2b2a-4677-925e-d0a9bb16032e" (UID: "53d05114-2b2a-4677-925e-d0a9bb16032e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:47.307212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.307109    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-model-cache" (OuterVolumeSpecName: "model-cache") pod "53d05114-2b2a-4677-925e-d0a9bb16032e" (UID: "53d05114-2b2a-4677-925e-d0a9bb16032e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:47.307212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.307128    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-home" (OuterVolumeSpecName: "home") pod "53d05114-2b2a-4677-925e-d0a9bb16032e" (UID: "53d05114-2b2a-4677-925e-d0a9bb16032e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:47.307212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.307184    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:47.307212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.307201    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:47.308941 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.308917    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53d05114-2b2a-4677-925e-d0a9bb16032e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "53d05114-2b2a-4677-925e-d0a9bb16032e" (UID: "53d05114-2b2a-4677-925e-d0a9bb16032e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:44:47.308941 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.308934    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-dshm" (OuterVolumeSpecName: "dshm") pod "53d05114-2b2a-4677-925e-d0a9bb16032e" (UID: "53d05114-2b2a-4677-925e-d0a9bb16032e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:44:47.309073 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.308999    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d05114-2b2a-4677-925e-d0a9bb16032e-kube-api-access-w78ck" (OuterVolumeSpecName: "kube-api-access-w78ck") pod "53d05114-2b2a-4677-925e-d0a9bb16032e" (UID: "53d05114-2b2a-4677-925e-d0a9bb16032e"). InnerVolumeSpecName "kube-api-access-w78ck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:44:47.407891 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.407854    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w78ck\" (UniqueName: \"kubernetes.io/projected/53d05114-2b2a-4677-925e-d0a9bb16032e-kube-api-access-w78ck\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:47.407891 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.407884    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:47.407891 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.407897    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53d05114-2b2a-4677-925e-d0a9bb16032e-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:47.408127 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.407907    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53d05114-2b2a-4677-925e-d0a9bb16032e-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:44:47.887617 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.887585    2565 generic.go:356] "Generic (PLEG): container finished" podID="53d05114-2b2a-4677-925e-d0a9bb16032e" containerID="072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae" exitCode=0
Jul 26 18:44:47.887817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.887636    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"
Jul 26 18:44:47.887817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.887675    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt" event={"ID":"53d05114-2b2a-4677-925e-d0a9bb16032e","Type":"ContainerDied","Data":"072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae"}
Jul 26 18:44:47.887817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.887731    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt" event={"ID":"53d05114-2b2a-4677-925e-d0a9bb16032e","Type":"ContainerDied","Data":"608a3d937e992570fb7d8e8acb8c50d1c232c3dcbafa9f3b3f2008363f31e436"}
Jul 26 18:44:47.887817 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.887753    2565 scope.go:117] "RemoveContainer" containerID="072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae"
Jul 26 18:44:47.896528 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.896512    2565 scope.go:117] "RemoveContainer" containerID="072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae"
Jul 26 18:44:47.896785 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:44:47.896767    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae\": container with ID starting with 072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae not found: ID does not exist" containerID="072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae"
Jul 26 18:44:47.896840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.896793    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae"} err="failed to get container status \"072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae\": rpc error: code = NotFound desc = could not find container \"072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae\": container with ID starting with 072838fc4c660282faa956b5fbc72d30d1f5ee1525502414bcae1604b4210eae not found: ID does not exist"
Jul 26 18:44:47.908101 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.908078    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:44:47.911047 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.911027    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"]
Jul 26 18:44:47.923088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:47.921490    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/sub-test-model-d112e4-kserve-85cb5f8685-9c7rt"]
Jul 26 18:44:48.320926 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:48.320890    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d05114-2b2a-4677-925e-d0a9bb16032e" path="/var/lib/kubelet/pods/53d05114-2b2a-4677-925e-d0a9bb16032e/volumes"
Jul 26 18:44:48.321407 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:44:48.321258    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" path="/var/lib/kubelet/pods/80d8e5b1-2a62-4bbb-98e7-34c719f08bf7/volumes"
Jul 26 18:45:00.180825 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.180787    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"]
Jul 26 18:45:00.181362 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.181312    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d05114-2b2a-4677-925e-d0a9bb16032e" containerName="main"
Jul 26 18:45:00.181362 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.181332    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d05114-2b2a-4677-925e-d0a9bb16032e" containerName="main"
Jul 26 18:45:00.181362 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.181345    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" containerName="main"
Jul 26 18:45:00.181362 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.181353    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" containerName="main"
Jul 26 18:45:00.181583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.181444    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d8e5b1-2a62-4bbb-98e7-34c719f08bf7" containerName="main"
Jul 26 18:45:00.181583 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.181461    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d05114-2b2a-4677-925e-d0a9bb16032e" containerName="main"
Jul 26 18:45:00.185450 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.185429    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"]
Jul 26 18:45:00.185567 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.185546    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"
Jul 26 18:45:00.188388 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.188371    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"]
Jul 26 18:45:00.188493 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.188459    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"
Jul 26 18:45:00.188685 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.188665    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:45:00.191496 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.191476    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"
Jul 26 18:45:00.195224 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.195201    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"]
Jul 26 18:45:00.197033 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.197008    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"
Jul 26 18:45:00.203186 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.203132    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"]
Jul 26 18:45:00.204611 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.204589    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"]
Jul 26 18:45:00.207239 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.207218    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"
Jul 26 18:45:00.239881 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.239857    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"
Jul 26 18:45:00.359737 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.359683    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"]
Jul 26 18:45:00.362745 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:45:00.362680    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07431173_3c12_4f26_8df6_70ec473aabe4.slice/crio-4feeecc4820699a6ada898dbe108e90bd552761e3ef2ac488d9d599e55026b77 WatchSource:0}: Error finding container 4feeecc4820699a6ada898dbe108e90bd552761e3ef2ac488d9d599e55026b77: Status 404 returned error can't find the container with id 4feeecc4820699a6ada898dbe108e90bd552761e3ef2ac488d9d599e55026b77
Jul 26 18:45:00.412152 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.412126    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"]
Jul 26 18:45:00.413613 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:45:00.413588    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617d082e_6d70_4f50_a2d1_c7ac5e410440.slice/crio-e3c82ab2c59eba85e78fc631aea6ea4b5a18ba3e2c97beaa8858e5e0fcebb416 WatchSource:0}: Error finding container e3c82ab2c59eba85e78fc631aea6ea4b5a18ba3e2c97beaa8858e5e0fcebb416: Status 404 returned error can't find the container with id e3c82ab2c59eba85e78fc631aea6ea4b5a18ba3e2c97beaa8858e5e0fcebb416
Jul 26 18:45:00.590207 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.590179    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"]
Jul 26 18:45:00.592536 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:45:00.592512    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d86370f_c438_439f_84b5_bd742717833f.slice/crio-0331ca6433801bc808e3a6062e248cc87cd0082fec27d5fe54e7b5697ba8350c WatchSource:0}: Error finding container 0331ca6433801bc808e3a6062e248cc87cd0082fec27d5fe54e7b5697ba8350c: Status 404 returned error can't find the container with id 0331ca6433801bc808e3a6062e248cc87cd0082fec27d5fe54e7b5697ba8350c
Jul 26 18:45:00.955042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.955010    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr" event={"ID":"7d86370f-c438-439f-84b5-bd742717833f","Type":"ContainerStarted","Data":"50d824dda323d1e00f3d25354dcd404ec5dfd9ff57dfa1cd85132640b1133475"}
Jul 26 18:45:00.955237 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.955069    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr" event={"ID":"7d86370f-c438-439f-84b5-bd742717833f","Type":"ContainerStarted","Data":"0331ca6433801bc808e3a6062e248cc87cd0082fec27d5fe54e7b5697ba8350c"}
Jul 26 18:45:00.956745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.956718    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs" event={"ID":"07431173-3c12-4f26-8df6-70ec473aabe4","Type":"ContainerStarted","Data":"73264a45e57f5a35d325401273b5eca1d27cbae691780ce5078829aca50fb06c"}
Jul 26 18:45:00.956885 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.956773    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs" event={"ID":"07431173-3c12-4f26-8df6-70ec473aabe4","Type":"ContainerStarted","Data":"4feeecc4820699a6ada898dbe108e90bd552761e3ef2ac488d9d599e55026b77"}
Jul 26 18:45:00.958275 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.958247    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l" event={"ID":"617d082e-6d70-4f50-a2d1-c7ac5e410440","Type":"ContainerStarted","Data":"7022ebc7117dfb33c6a2512754abf00b1329219fe26816945f2ef6a5e0d97f90"}
Jul 26 18:45:00.958389 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.958303    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l" event={"ID":"617d082e-6d70-4f50-a2d1-c7ac5e410440","Type":"ContainerStarted","Data":"e3c82ab2c59eba85e78fc631aea6ea4b5a18ba3e2c97beaa8858e5e0fcebb416"}
Jul 26 18:45:00.974974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.974915    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr" podStartSLOduration=0.974890968 podStartE2EDuration="974.890968ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:00.970894474 +0000 UTC m=+2855.160924235" watchObservedRunningTime="2026-07-26 18:45:00.974890968 +0000 UTC m=+2855.164920742"
Jul 26 18:45:00.996228 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:00.996175    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l" podStartSLOduration=0.996161411 podStartE2EDuration="996.161411ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:00.993452922 +0000 UTC m=+2855.183482677" watchObservedRunningTime="2026-07-26 18:45:00.996161411 +0000 UTC m=+2855.186191164"
Jul 26 18:45:01.014275 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.014224    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs" podStartSLOduration=1.014209549 podStartE2EDuration="1.014209549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:01.011402882 +0000 UTC m=+2855.201432635" watchObservedRunningTime="2026-07-26 18:45:01.014209549 +0000 UTC m=+2855.204239303"
Jul 26 18:45:01.965312 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.965284    2565 generic.go:356] "Generic (PLEG): container finished" podID="7d86370f-c438-439f-84b5-bd742717833f" containerID="50d824dda323d1e00f3d25354dcd404ec5dfd9ff57dfa1cd85132640b1133475" exitCode=0
Jul 26 18:45:01.965881 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.965363    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr" event={"ID":"7d86370f-c438-439f-84b5-bd742717833f","Type":"ContainerDied","Data":"50d824dda323d1e00f3d25354dcd404ec5dfd9ff57dfa1cd85132640b1133475"}
Jul 26 18:45:01.966920 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.966896    2565 generic.go:356] "Generic (PLEG): container finished" podID="07431173-3c12-4f26-8df6-70ec473aabe4" containerID="73264a45e57f5a35d325401273b5eca1d27cbae691780ce5078829aca50fb06c" exitCode=0
Jul 26 18:45:01.967029 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.966972    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs" event={"ID":"07431173-3c12-4f26-8df6-70ec473aabe4","Type":"ContainerDied","Data":"73264a45e57f5a35d325401273b5eca1d27cbae691780ce5078829aca50fb06c"}
Jul 26 18:45:01.968479 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.968458    2565 generic.go:356] "Generic (PLEG): container finished" podID="617d082e-6d70-4f50-a2d1-c7ac5e410440" containerID="7022ebc7117dfb33c6a2512754abf00b1329219fe26816945f2ef6a5e0d97f90" exitCode=0
Jul 26 18:45:01.968605 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:01.968485    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l" event={"ID":"617d082e-6d70-4f50-a2d1-c7ac5e410440","Type":"ContainerDied","Data":"7022ebc7117dfb33c6a2512754abf00b1329219fe26816945f2ef6a5e0d97f90"}
Jul 26 18:45:02.121523 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.121491    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"]
Jul 26 18:45:02.124722 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.124687    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.127489 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.127464    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"openshift-service-ca.crt\""
Jul 26 18:45:02.127654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.127631    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"default-dockercfg-678bm\""
Jul 26 18:45:02.127812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.127631    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"rate-test-model-37e93d-kserve-self-signed-certs\""
Jul 26 18:45:02.127812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.127668    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-37e93d\"/\"kube-root-ca.crt\""
Jul 26 18:45:02.127973 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.127952    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-dshm\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.128014 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.127989    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-tmp-dir\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.128072 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.128058    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhlhd\" (UniqueName: \"kubernetes.io/projected/2ed13ad8-2949-413e-9ca8-9276d3431236-kube-api-access-jhlhd\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.128119 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.128097    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed13ad8-2949-413e-9ca8-9276d3431236-tls-certs\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.128157 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.128144    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-home\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.128229 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.128194    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-model-cache\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.134383 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.134363    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"]
Jul 26 18:45:02.228807 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.228777    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhlhd\" (UniqueName: \"kubernetes.io/projected/2ed13ad8-2949-413e-9ca8-9276d3431236-kube-api-access-jhlhd\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.228996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.228813    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed13ad8-2949-413e-9ca8-9276d3431236-tls-certs\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.228996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.228844    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-home\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.228996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.228876    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-model-cache\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.228996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.228910    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-dshm\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.228996 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.228926    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-tmp-dir\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.229373 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.229351    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-tmp-dir\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.229454 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.229381    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-model-cache\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.229454 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.229391    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-home\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.231219 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.231197    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-dshm\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.231461 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.231446    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed13ad8-2949-413e-9ca8-9276d3431236-tls-certs\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.237521 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.237498    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhlhd\" (UniqueName: \"kubernetes.io/projected/2ed13ad8-2949-413e-9ca8-9276d3431236-kube-api-access-jhlhd\") pod \"rate-test-model-37e93d-kserve-64549fdb87-r95xh\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") " pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.435342 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.435306    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:02.572669 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.572643    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"]
Jul 26 18:45:02.573976 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:45:02.573949    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ed13ad8_2949_413e_9ca8_9276d3431236.slice/crio-2383eb3a80227259cc20c8debf4e968ff5cfb8b9f70b8585667cadbd7114848e WatchSource:0}: Error finding container 2383eb3a80227259cc20c8debf4e968ff5cfb8b9f70b8585667cadbd7114848e: Status 404 returned error can't find the container with id 2383eb3a80227259cc20c8debf4e968ff5cfb8b9f70b8585667cadbd7114848e
Jul 26 18:45:02.975909 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.975871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh" event={"ID":"2ed13ad8-2949-413e-9ca8-9276d3431236","Type":"ContainerStarted","Data":"f013f3f18f1bf81870477ef5f16ddb5b9c4962dd2b55dd1effb4b937334d590f"}
Jul 26 18:45:02.976377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.975918    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh" event={"ID":"2ed13ad8-2949-413e-9ca8-9276d3431236","Type":"ContainerStarted","Data":"2383eb3a80227259cc20c8debf4e968ff5cfb8b9f70b8585667cadbd7114848e"}
Jul 26 18:45:02.997324 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:02.997223    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh" podStartSLOduration=0.997199965 podStartE2EDuration="997.199965ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:02.99647233 +0000 UTC m=+2857.186502076" watchObservedRunningTime="2026-07-26 18:45:02.997199965 +0000 UTC m=+2857.187229721"
Jul 26 18:45:03.126923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.126893    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"
Jul 26 18:45:03.129763 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.129737    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"
Jul 26 18:45:03.171345 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.171317    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"
Jul 26 18:45:03.982167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.982136    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"
Jul 26 18:45:03.982655 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.982181    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l" event={"ID":"617d082e-6d70-4f50-a2d1-c7ac5e410440","Type":"ContainerDied","Data":"e3c82ab2c59eba85e78fc631aea6ea4b5a18ba3e2c97beaa8858e5e0fcebb416"}
Jul 26 18:45:03.982655 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.982245    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3c82ab2c59eba85e78fc631aea6ea4b5a18ba3e2c97beaa8858e5e0fcebb416"
Jul 26 18:45:03.983796 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.983775    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr"
Jul 26 18:45:03.983980 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.983768    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29751525-7cfnr" event={"ID":"7d86370f-c438-439f-84b5-bd742717833f","Type":"ContainerDied","Data":"0331ca6433801bc808e3a6062e248cc87cd0082fec27d5fe54e7b5697ba8350c"}
Jul 26 18:45:03.983980 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.983922    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0331ca6433801bc808e3a6062e248cc87cd0082fec27d5fe54e7b5697ba8350c"
Jul 26 18:45:03.985098 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.985075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs" event={"ID":"07431173-3c12-4f26-8df6-70ec473aabe4","Type":"ContainerDied","Data":"4feeecc4820699a6ada898dbe108e90bd552761e3ef2ac488d9d599e55026b77"}
Jul 26 18:45:03.985098 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.985093    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"
Jul 26 18:45:03.985241 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:03.985121    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4feeecc4820699a6ada898dbe108e90bd552761e3ef2ac488d9d599e55026b77"
Jul 26 18:45:05.414581 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:05.414548    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:45:12.435644 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:12.435600    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:12.435644 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:12.435651    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:12.448149 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:12.448126    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:13.042957 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:13.042928    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:45:33.926316 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926276    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"]
Jul 26 18:45:33.926853 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926814    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07431173-3c12-4f26-8df6-70ec473aabe4" containerName="cleanup"
Jul 26 18:45:33.926853 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926837    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="07431173-3c12-4f26-8df6-70ec473aabe4" containerName="cleanup"
Jul 26 18:45:33.926977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926861    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="617d082e-6d70-4f50-a2d1-c7ac5e410440" containerName="cleanup"
Jul 26 18:45:33.926977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926870    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="617d082e-6d70-4f50-a2d1-c7ac5e410440" containerName="cleanup"
Jul 26 18:45:33.926977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926879    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d86370f-c438-439f-84b5-bd742717833f" containerName="cleanup"
Jul 26 18:45:33.926977 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926888    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d86370f-c438-439f-84b5-bd742717833f" containerName="cleanup"
Jul 26 18:45:33.927167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.926994    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="617d082e-6d70-4f50-a2d1-c7ac5e410440" containerName="cleanup"
Jul 26 18:45:33.927167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.927009    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="07431173-3c12-4f26-8df6-70ec473aabe4" containerName="cleanup"
Jul 26 18:45:33.927167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.927023    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d86370f-c438-439f-84b5-bd742717833f" containerName="cleanup"
Jul 26 18:45:33.931177 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.931156    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:33.935011 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.934988    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"kube-root-ca.crt\""
Jul 26 18:45:33.935138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.935021    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"default-dockercfg-5h9xv\""
Jul 26 18:45:33.935138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.935035    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"openshift-service-ca.crt\""
Jul 26 18:45:33.935138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.935036    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d112e4\"/\"rate-test-model-d112e4-kserve-self-signed-certs\""
Jul 26 18:45:33.939851 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.939831    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"]
Jul 26 18:45:33.997363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.997322    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-home\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:33.997363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.997366    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tls-certs\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:33.997587 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.997462    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-model-cache\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:33.997587 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.997500    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-dshm\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:33.997587 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.997564    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqvkt\" (UniqueName: \"kubernetes.io/projected/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-kube-api-access-mqvkt\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:33.997745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:33.997596    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tmp-dir\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099057 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099021    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-model-cache\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099263 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099067    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-dshm\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099263 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099203    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqvkt\" (UniqueName: \"kubernetes.io/projected/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-kube-api-access-mqvkt\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099263 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099241    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tmp-dir\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099340    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-home\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099449 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099362    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tls-certs\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099442    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-model-cache\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099646 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099624    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-home\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.099646 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.099639    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tmp-dir\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.101329 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.101308    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-dshm\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.101648 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.101629    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tls-certs\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.106899 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.106882    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqvkt\" (UniqueName: \"kubernetes.io/projected/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-kube-api-access-mqvkt\") pod \"rate-test-model-d112e4-kserve-65cdffc579-f8ppz\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") " pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.241589 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.241553    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:34.374951 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:34.374916    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"]
Jul 26 18:45:34.377962 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:45:34.377936    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bd3eaa7_bc53_4f3c_8ef4_ede81ebe3110.slice/crio-64e018e5b289c397475da02ef84eb3eeaa921fd964aeda15c907386a635558cf WatchSource:0}: Error finding container 64e018e5b289c397475da02ef84eb3eeaa921fd964aeda15c907386a635558cf: Status 404 returned error can't find the container with id 64e018e5b289c397475da02ef84eb3eeaa921fd964aeda15c907386a635558cf
Jul 26 18:45:35.142690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:35.142651    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz" event={"ID":"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110","Type":"ContainerStarted","Data":"1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9"}
Jul 26 18:45:35.142690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:35.142709    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz" event={"ID":"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110","Type":"ContainerStarted","Data":"64e018e5b289c397475da02ef84eb3eeaa921fd964aeda15c907386a635558cf"}
Jul 26 18:45:35.162005 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:35.161947    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz" podStartSLOduration=2.161931479 podStartE2EDuration="2.161931479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:45:35.159726562 +0000 UTC m=+2889.349756327" watchObservedRunningTime="2026-07-26 18:45:35.161931479 +0000 UTC m=+2889.351961232"
Jul 26 18:45:37.612825 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:37.612787    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:45:44.241761 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:44.241725    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:44.241761 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:44.241773    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:44.254363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:44.254338    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:45:45.206709 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:45:45.206679    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:46:10.711790 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:10.711752    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:46:30.839978 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:30.839937    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:46:37.275029 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:37.274994    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"]
Jul 26 18:46:38.105094 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:38.105061    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"]
Jul 26 18:46:45.025145 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:45.025105    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:46:52.294251 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.294146    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh" podUID="2ed13ad8-2949-413e-9ca8-9276d3431236" containerName="main" containerID="cri-o://f013f3f18f1bf81870477ef5f16ddb5b9c4962dd2b55dd1effb4b937334d590f" gracePeriod=45
Jul 26 18:46:52.533006 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.532980    2565 generic.go:356] "Generic (PLEG): container finished" podID="2ed13ad8-2949-413e-9ca8-9276d3431236" containerID="f013f3f18f1bf81870477ef5f16ddb5b9c4962dd2b55dd1effb4b937334d590f" exitCode=0
Jul 26 18:46:52.533149 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.533062    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh" event={"ID":"2ed13ad8-2949-413e-9ca8-9276d3431236","Type":"ContainerDied","Data":"f013f3f18f1bf81870477ef5f16ddb5b9c4962dd2b55dd1effb4b937334d590f"}
Jul 26 18:46:52.548931 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.548861    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:46:52.661487 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661454    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed13ad8-2949-413e-9ca8-9276d3431236-tls-certs\") pod \"2ed13ad8-2949-413e-9ca8-9276d3431236\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") "
Jul 26 18:46:52.661732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661509    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhlhd\" (UniqueName: \"kubernetes.io/projected/2ed13ad8-2949-413e-9ca8-9276d3431236-kube-api-access-jhlhd\") pod \"2ed13ad8-2949-413e-9ca8-9276d3431236\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") "
Jul 26 18:46:52.661732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661536    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-dshm\") pod \"2ed13ad8-2949-413e-9ca8-9276d3431236\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") "
Jul 26 18:46:52.661732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661572    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-tmp-dir\") pod \"2ed13ad8-2949-413e-9ca8-9276d3431236\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") "
Jul 26 18:46:52.661732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661623    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-model-cache\") pod \"2ed13ad8-2949-413e-9ca8-9276d3431236\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") "
Jul 26 18:46:52.661732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661653    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-home\") pod \"2ed13ad8-2949-413e-9ca8-9276d3431236\" (UID: \"2ed13ad8-2949-413e-9ca8-9276d3431236\") "
Jul 26 18:46:52.662015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661926    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ed13ad8-2949-413e-9ca8-9276d3431236" (UID: "2ed13ad8-2949-413e-9ca8-9276d3431236"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:52.662015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661944    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ed13ad8-2949-413e-9ca8-9276d3431236" (UID: "2ed13ad8-2949-413e-9ca8-9276d3431236"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:52.662015 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.661954    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-home" (OuterVolumeSpecName: "home") pod "2ed13ad8-2949-413e-9ca8-9276d3431236" (UID: "2ed13ad8-2949-413e-9ca8-9276d3431236"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:52.663867 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.663839    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed13ad8-2949-413e-9ca8-9276d3431236-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ed13ad8-2949-413e-9ca8-9276d3431236" (UID: "2ed13ad8-2949-413e-9ca8-9276d3431236"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:46:52.663972 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.663845    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-dshm" (OuterVolumeSpecName: "dshm") pod "2ed13ad8-2949-413e-9ca8-9276d3431236" (UID: "2ed13ad8-2949-413e-9ca8-9276d3431236"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:52.663972 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.663840    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed13ad8-2949-413e-9ca8-9276d3431236-kube-api-access-jhlhd" (OuterVolumeSpecName: "kube-api-access-jhlhd") pod "2ed13ad8-2949-413e-9ca8-9276d3431236" (UID: "2ed13ad8-2949-413e-9ca8-9276d3431236"). InnerVolumeSpecName "kube-api-access-jhlhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:46:52.762545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.762493    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:52.762545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.762537    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:52.762545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.762547    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:52.762545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.762556    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ed13ad8-2949-413e-9ca8-9276d3431236-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:52.762545 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.762565    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhlhd\" (UniqueName: \"kubernetes.io/projected/2ed13ad8-2949-413e-9ca8-9276d3431236-kube-api-access-jhlhd\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:52.762945 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:52.762574    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ed13ad8-2949-413e-9ca8-9276d3431236-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.123455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.123405    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz" podUID="6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" containerName="main" containerID="cri-o://1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9" gracePeriod=45
Jul 26 18:46:53.379659 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.379589    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:46:53.469435 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469395    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqvkt\" (UniqueName: \"kubernetes.io/projected/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-kube-api-access-mqvkt\") pod \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") "
Jul 26 18:46:53.469615 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469461    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-dshm\") pod \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") "
Jul 26 18:46:53.469615 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469483    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-home\") pod \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") "
Jul 26 18:46:53.469615 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469501    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-model-cache\") pod \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") "
Jul 26 18:46:53.469615 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469533    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tmp-dir\") pod \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") "
Jul 26 18:46:53.469615 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469564    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tls-certs\") pod \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\" (UID: \"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110\") "
Jul 26 18:46:53.469924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469818    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-home" (OuterVolumeSpecName: "home") pod "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" (UID: "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:53.469924 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.469832    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-model-cache" (OuterVolumeSpecName: "model-cache") pod "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" (UID: "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:53.470158 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.470135    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" (UID: "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:53.471710 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.471677    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-dshm" (OuterVolumeSpecName: "dshm") pod "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" (UID: "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:46:53.471796 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.471764    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-kube-api-access-mqvkt" (OuterVolumeSpecName: "kube-api-access-mqvkt") pod "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" (UID: "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110"). InnerVolumeSpecName "kube-api-access-mqvkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:46:53.472021 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.471990    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" (UID: "6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:46:53.538925 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.538890    2565 generic.go:356] "Generic (PLEG): container finished" podID="6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" containerID="1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9" exitCode=0
Jul 26 18:46:53.539139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.538934    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"
Jul 26 18:46:53.539139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.538979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz" event={"ID":"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110","Type":"ContainerDied","Data":"1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9"}
Jul 26 18:46:53.539139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.539025    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz" event={"ID":"6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110","Type":"ContainerDied","Data":"64e018e5b289c397475da02ef84eb3eeaa921fd964aeda15c907386a635558cf"}
Jul 26 18:46:53.539139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.539046    2565 scope.go:117] "RemoveContainer" containerID="1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9"
Jul 26 18:46:53.540447 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.540416    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh" event={"ID":"2ed13ad8-2949-413e-9ca8-9276d3431236","Type":"ContainerDied","Data":"2383eb3a80227259cc20c8debf4e968ff5cfb8b9f70b8585667cadbd7114848e"}
Jul 26 18:46:53.540447 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.540440    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"
Jul 26 18:46:53.548998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.548910    2565 scope.go:117] "RemoveContainer" containerID="1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9"
Jul 26 18:46:53.549245 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:46:53.549216    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9\": container with ID starting with 1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9 not found: ID does not exist" containerID="1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9"
Jul 26 18:46:53.549445 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.549256    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9"} err="failed to get container status \"1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9\": rpc error: code = NotFound desc = could not find container \"1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9\": container with ID starting with 1b1ee0f4870dc7582d50eb0ef860359bd102472a63e5fae6dbea041046aea4d9 not found: ID does not exist"
Jul 26 18:46:53.549445 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.549274    2565 scope.go:117] "RemoveContainer" containerID="f013f3f18f1bf81870477ef5f16ddb5b9c4962dd2b55dd1effb4b937334d590f"
Jul 26 18:46:53.563432 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.563404    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"]
Jul 26 18:46:53.566920 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.566893    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-37e93d/rate-test-model-37e93d-kserve-64549fdb87-r95xh"]
Jul 26 18:46:53.571013 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.570989    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqvkt\" (UniqueName: \"kubernetes.io/projected/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-kube-api-access-mqvkt\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.571013 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.571011    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.571189 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.571020    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.571189 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.571028    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.571189 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.571037    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.571189 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.571045    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:46:53.577345 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.577318    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"]
Jul 26 18:46:53.580807 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:53.580786    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d112e4/rate-test-model-d112e4-kserve-65cdffc579-f8ppz"]
Jul 26 18:46:54.319075 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:54.319044    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed13ad8-2949-413e-9ca8-9276d3431236" path="/var/lib/kubelet/pods/2ed13ad8-2949-413e-9ca8-9276d3431236/volumes"
Jul 26 18:46:54.319426 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:46:54.319413    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" path="/var/lib/kubelet/pods/6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110/volumes"
Jul 26 18:47:05.172521 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172483    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"]
Jul 26 18:47:05.173018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172843    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" containerName="main"
Jul 26 18:47:05.173018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172856    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" containerName="main"
Jul 26 18:47:05.173018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172874    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed13ad8-2949-413e-9ca8-9276d3431236" containerName="main"
Jul 26 18:47:05.173018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172879    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed13ad8-2949-413e-9ca8-9276d3431236" containerName="main"
Jul 26 18:47:05.173018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172930    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed13ad8-2949-413e-9ca8-9276d3431236" containerName="main"
Jul 26 18:47:05.173018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.172940    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bd3eaa7-bc53-4f3c-8ef4-ede81ebe3110" containerName="main"
Jul 26 18:47:05.177024 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.177005    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.179917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.179894    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-33964c\""
Jul 26 18:47:05.180034 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.179945    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-33964c-dockercfg-pvhm6\""
Jul 26 18:47:05.185370 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.185351    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"]
Jul 26 18:47:05.197105 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.197076    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"]
Jul 26 18:47:05.202845 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.201315    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.210802 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.210642    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"]
Jul 26 18:47:05.257836 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.257794    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.257836 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.257832    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.258098 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.257908    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djl6k\" (UniqueName: \"kubernetes.io/projected/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-kube-api-access-djl6k\") pod \"payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.258098 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.258008    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tsrb\" (UniqueName: \"kubernetes.io/projected/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-kube-api-access-8tsrb\") pod \"payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.358381 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.358345    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tsrb\" (UniqueName: \"kubernetes.io/projected/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-kube-api-access-8tsrb\") pod \"payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.358576 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.358407    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.358576 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.358434    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.358576 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.358483    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djl6k\" (UniqueName: \"kubernetes.io/projected/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-kube-api-access-djl6k\") pod \"payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.359244 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.359222    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.359308 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.359240    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.366910 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.366881    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djl6k\" (UniqueName: \"kubernetes.io/projected/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-kube-api-access-djl6k\") pod \"payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.367055 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.366936    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tsrb\" (UniqueName: \"kubernetes.io/projected/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-kube-api-access-8tsrb\") pod \"payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.488826 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.488793    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:05.515084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.515056    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:05.635536 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.635448    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"]
Jul 26 18:47:05.637629 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:47:05.637600    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb01b4f6_3bdc_4843_83c9_d5833cbec96d.slice/crio-5344f0fa873898d20cd2a66beabd1d60c6d84fb255d5a48136d1f6a813a81802 WatchSource:0}: Error finding container 5344f0fa873898d20cd2a66beabd1d60c6d84fb255d5a48136d1f6a813a81802: Status 404 returned error can't find the container with id 5344f0fa873898d20cd2a66beabd1d60c6d84fb255d5a48136d1f6a813a81802
Jul 26 18:47:05.662894 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:05.662871    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"]
Jul 26 18:47:05.669910 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:47:05.669884    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e0d3c4_020d_4a0b_81d1_4a77a7371662.slice/crio-230af2511e9c6144e19e70de41c708d32bec7e64bf424c0caf9ebd842ee2616c WatchSource:0}: Error finding container 230af2511e9c6144e19e70de41c708d32bec7e64bf424c0caf9ebd842ee2616c: Status 404 returned error can't find the container with id 230af2511e9c6144e19e70de41c708d32bec7e64bf424c0caf9ebd842ee2616c
Jul 26 18:47:06.618958 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.618919    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr" event={"ID":"e0e0d3c4-020d-4a0b-81d1-4a77a7371662","Type":"ContainerStarted","Data":"83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6"}
Jul 26 18:47:06.619462 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.618979    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr" event={"ID":"e0e0d3c4-020d-4a0b-81d1-4a77a7371662","Type":"ContainerStarted","Data":"230af2511e9c6144e19e70de41c708d32bec7e64bf424c0caf9ebd842ee2616c"}
Jul 26 18:47:06.619462 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.619025    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:06.620512 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.620483    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62" event={"ID":"bb01b4f6-3bdc-4843-83c9-d5833cbec96d","Type":"ContainerStarted","Data":"c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf"}
Jul 26 18:47:06.620573 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.620534    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:06.620573 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.620550    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62" event={"ID":"bb01b4f6-3bdc-4843-83c9-d5833cbec96d","Type":"ContainerStarted","Data":"5344f0fa873898d20cd2a66beabd1d60c6d84fb255d5a48136d1f6a813a81802"}
Jul 26 18:47:06.636182 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.636118    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr" podStartSLOduration=1.636098389 podStartE2EDuration="1.636098389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:06.634746102 +0000 UTC m=+2980.824775857" watchObservedRunningTime="2026-07-26 18:47:06.636098389 +0000 UTC m=+2980.826128143"
Jul 26 18:47:06.650623 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:06.650575    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62" podStartSLOduration=1.650558094 podStartE2EDuration="1.650558094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:06.64956726 +0000 UTC m=+2980.839597018" watchObservedRunningTime="2026-07-26 18:47:06.650558094 +0000 UTC m=+2980.840587849"
Jul 26 18:47:08.119925 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:08.119894    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:47:17.628127 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:17.628094    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:47:17.628559 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:17.628171    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:47:23.917970 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.917931    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"]
Jul 26 18:47:23.921535 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.921518    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:23.924465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.924438    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f6bce0\""
Jul 26 18:47:23.924598 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.924464    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-f6bce0-dockercfg-xfhrn\""
Jul 26 18:47:23.932208 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.932180    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"]
Jul 26 18:47:23.937694 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.937668    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"]
Jul 26 18:47:23.943054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.943021    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:23.952672 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:23.952643    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"]
Jul 26 18:47:24.015227 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.015185    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn5fg\" (UniqueName: \"kubernetes.io/projected/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-kube-api-access-gn5fg\") pod \"payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.015438 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.015339    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.015438 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.015408    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f611942-fe45-4177-9447-5968b6e942c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.015573 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.015443    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5svl\" (UniqueName: \"kubernetes.io/projected/4f611942-fe45-4177-9447-5968b6e942c7-kube-api-access-h5svl\") pod \"payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.116875 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.116829    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.117054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.116884    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f611942-fe45-4177-9447-5968b6e942c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.117054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.116911    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5svl\" (UniqueName: \"kubernetes.io/projected/4f611942-fe45-4177-9447-5968b6e942c7-kube-api-access-h5svl\") pod \"payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.117054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.116942    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn5fg\" (UniqueName: \"kubernetes.io/projected/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-kube-api-access-gn5fg\") pod \"payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.117582 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.117561    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.117754 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.117733    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f611942-fe45-4177-9447-5968b6e942c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.125842 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.125810    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5svl\" (UniqueName: \"kubernetes.io/projected/4f611942-fe45-4177-9447-5968b6e942c7-kube-api-access-h5svl\") pod \"payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.125842 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.125822    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn5fg\" (UniqueName: \"kubernetes.io/projected/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-kube-api-access-gn5fg\") pod \"payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.233479 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.233437    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.258593 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.258562    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.379861 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.379829    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"]
Jul 26 18:47:24.381910 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:47:24.381875    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f611942_fe45_4177_9447_5968b6e942c7.slice/crio-3b2b5afc88433370883bbee5a328f55dfcc24244dafae1d9968c43afe5e6107d WatchSource:0}: Error finding container 3b2b5afc88433370883bbee5a328f55dfcc24244dafae1d9968c43afe5e6107d: Status 404 returned error can't find the container with id 3b2b5afc88433370883bbee5a328f55dfcc24244dafae1d9968c43afe5e6107d
Jul 26 18:47:24.610483 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.610460    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"]
Jul 26 18:47:24.612337 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:47:24.612304    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb5975f_a6de_4a71_8c8e_435b68c0d26c.slice/crio-9b848d78147d777c39d3a9c27196b65136c848ec529cc514475e99acc81be5e5 WatchSource:0}: Error finding container 9b848d78147d777c39d3a9c27196b65136c848ec529cc514475e99acc81be5e5: Status 404 returned error can't find the container with id 9b848d78147d777c39d3a9c27196b65136c848ec529cc514475e99acc81be5e5
Jul 26 18:47:24.717912 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.717877    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2" event={"ID":"4f611942-fe45-4177-9447-5968b6e942c7","Type":"ContainerStarted","Data":"76d9c0a73988a7877f89f75e523cf34fa1e55143ecb7493df35444c3cedec2f7"}
Jul 26 18:47:24.718112 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.717949    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:24.718112 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.717977    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2" event={"ID":"4f611942-fe45-4177-9447-5968b6e942c7","Type":"ContainerStarted","Data":"3b2b5afc88433370883bbee5a328f55dfcc24244dafae1d9968c43afe5e6107d"}
Jul 26 18:47:24.719513 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.719484    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw" event={"ID":"3cb5975f-a6de-4a71-8c8e-435b68c0d26c","Type":"ContainerStarted","Data":"b1512591157d89f03cdf33471ed549f803317a6cb131772d7e2fe6f391da0cd4"}
Jul 26 18:47:24.719613 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.719540    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:24.719613 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.719568    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw" event={"ID":"3cb5975f-a6de-4a71-8c8e-435b68c0d26c","Type":"ContainerStarted","Data":"9b848d78147d777c39d3a9c27196b65136c848ec529cc514475e99acc81be5e5"}
Jul 26 18:47:24.734278 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.734230    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2" podStartSLOduration=1.734211852 podStartE2EDuration="1.734211852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:24.733476249 +0000 UTC m=+2998.923506004" watchObservedRunningTime="2026-07-26 18:47:24.734211852 +0000 UTC m=+2998.924241606"
Jul 26 18:47:24.749859 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:24.749811    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw" podStartSLOduration=1.74979546 podStartE2EDuration="1.74979546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:24.748992249 +0000 UTC m=+2998.939022002" watchObservedRunningTime="2026-07-26 18:47:24.74979546 +0000 UTC m=+2998.939825215"
Jul 26 18:47:32.710596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:32.710560    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:47:35.727266 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:35.727229    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:47:35.727739 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:35.727316    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:47:44.762620 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:44.762590    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw_3cb5975f-a6de-4a71-8c8e-435b68c0d26c/payload-processing/0.log"
Jul 26 18:47:59.932878 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.932840    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"]
Jul 26 18:47:59.937106 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.937083    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:47:59.939923 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.939901    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-33964c\"/\"kube-root-ca.crt\""
Jul 26 18:47:59.940023 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.939904    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-33964c\"/\"openshift-service-ca.crt\""
Jul 26 18:47:59.941071 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.941055    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-33964c\"/\"ipp-route-33964c-kserve-self-signed-certs\""
Jul 26 18:47:59.941147 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.941112    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-33964c\"/\"default-dockercfg-fvwk9\""
Jul 26 18:47:59.946834 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:47:59.946816    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"]
Jul 26 18:48:00.019751 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.019690    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-dshm\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.019974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.019788    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-home\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.019974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.019825    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg8lk\" (UniqueName: \"kubernetes.io/projected/144c575f-5f78-48eb-8dfe-159d23d5fc56-kube-api-access-kg8lk\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.019974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.019861    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144c575f-5f78-48eb-8dfe-159d23d5fc56-tls-certs\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.019974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.019928    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-tmp-dir\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.019974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.019955    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-model-cache\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121036 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.120998    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-tmp-dir\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121036 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121038    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-model-cache\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121307 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121060    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-dshm\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121307 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121103    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-home\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121307 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121140    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg8lk\" (UniqueName: \"kubernetes.io/projected/144c575f-5f78-48eb-8dfe-159d23d5fc56-kube-api-access-kg8lk\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121307 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121180    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144c575f-5f78-48eb-8dfe-159d23d5fc56-tls-certs\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121523 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121481    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-tmp-dir\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121570 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121518    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-model-cache\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.121605 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.121579    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-home\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.123457 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.123436    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-dshm\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.123643 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.123628    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144c575f-5f78-48eb-8dfe-159d23d5fc56-tls-certs\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.130043 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.130012    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg8lk\" (UniqueName: \"kubernetes.io/projected/144c575f-5f78-48eb-8dfe-159d23d5fc56-kube-api-access-kg8lk\") pod \"ipp-route-33964c-kserve-bbf57d48c-lhrv2\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") " pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.247499 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.247455    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:00.394002 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.393976    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"]
Jul 26 18:48:00.395800 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:48:00.395766    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod144c575f_5f78_48eb_8dfe_159d23d5fc56.slice/crio-933032ff4e9bf17065ce3ca03bc291f41f5dae4e782496f52a0de6109c1aa7d2 WatchSource:0}: Error finding container 933032ff4e9bf17065ce3ca03bc291f41f5dae4e782496f52a0de6109c1aa7d2: Status 404 returned error can't find the container with id 933032ff4e9bf17065ce3ca03bc291f41f5dae4e782496f52a0de6109c1aa7d2
Jul 26 18:48:00.913813 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.913777    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2" event={"ID":"144c575f-5f78-48eb-8dfe-159d23d5fc56","Type":"ContainerStarted","Data":"ef88440edb1bb738a50c3f6cf709d31d612731ffdb0f3957036e018d04496e00"}
Jul 26 18:48:00.914010 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.913827    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2" event={"ID":"144c575f-5f78-48eb-8dfe-159d23d5fc56","Type":"ContainerStarted","Data":"933032ff4e9bf17065ce3ca03bc291f41f5dae4e782496f52a0de6109c1aa7d2"}
Jul 26 18:48:00.933898 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:00.933847    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2" podStartSLOduration=1.9338293659999999 podStartE2EDuration="1.933829366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:48:00.932993584 +0000 UTC m=+3035.123023341" watchObservedRunningTime="2026-07-26 18:48:00.933829366 +0000 UTC m=+3035.123859154"
Jul 26 18:48:03.910816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:03.910780    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:48:10.248567 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:10.248528    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:10.249071 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:10.248583    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:10.261249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:10.261222    2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:10.976881 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:10.976852    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:48:29.421209 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:29.421127    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:48:39.649801 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:39.649772    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr_e0e0d3c4-020d-4a0b-81d1-4a77a7371662/payload-processing/0.log"
Jul 26 18:48:39.883852 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:39.883821    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw_3cb5975f-a6de-4a71-8c8e-435b68c0d26c/payload-processing/0.log"
Jul 26 18:48:44.353456 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.353423    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"]
Jul 26 18:48:44.356968 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.356951    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.359778 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.359757    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-786276\""
Jul 26 18:48:44.359944 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.359760    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-786276-dockercfg-ngmpz\""
Jul 26 18:48:44.369754 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.369730    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"]
Jul 26 18:48:44.407011 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.406973    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"]
Jul 26 18:48:44.410363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.410345    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.421623 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.421598    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"]
Jul 26 18:48:44.500030 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.499996    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psdf7\" (UniqueName: \"kubernetes.io/projected/2df467ea-0597-4792-bf7d-5b60df1d3ff8-kube-api-access-psdf7\") pod \"payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.500212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.500111    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8rr7\" (UniqueName: \"kubernetes.io/projected/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-kube-api-access-g8rr7\") pod \"payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.500212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.500144    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df467ea-0597-4792-bf7d-5b60df1d3ff8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.500212 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.500163    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.601045 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.601010    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psdf7\" (UniqueName: \"kubernetes.io/projected/2df467ea-0597-4792-bf7d-5b60df1d3ff8-kube-api-access-psdf7\") pod \"payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.601215 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.601089    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8rr7\" (UniqueName: \"kubernetes.io/projected/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-kube-api-access-g8rr7\") pod \"payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.601215 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.601122    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df467ea-0597-4792-bf7d-5b60df1d3ff8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.601215 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.601145    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.601890 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.601872    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.601938 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.601871    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df467ea-0597-4792-bf7d-5b60df1d3ff8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.609142 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.609084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psdf7\" (UniqueName: \"kubernetes.io/projected/2df467ea-0597-4792-bf7d-5b60df1d3ff8-kube-api-access-psdf7\") pod \"payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.609142 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.609113    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8rr7\" (UniqueName: \"kubernetes.io/projected/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-kube-api-access-g8rr7\") pod \"payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.667065 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.667032    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:44.720669 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.720614    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:44.806535 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.806505    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"]
Jul 26 18:48:44.808756 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:48:44.808730    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e28df0_e5dd_4c8b_a2aa_fd563e206b85.slice/crio-e43c71bc22e5b33312ed67c1ae5ae83a79eff3a44b4811a71e43a997e0737a39 WatchSource:0}: Error finding container e43c71bc22e5b33312ed67c1ae5ae83a79eff3a44b4811a71e43a997e0737a39: Status 404 returned error can't find the container with id e43c71bc22e5b33312ed67c1ae5ae83a79eff3a44b4811a71e43a997e0737a39
Jul 26 18:48:44.875539 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:44.875509    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"]
Jul 26 18:48:44.876560 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:48:44.876534    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df467ea_0597_4792_bf7d_5b60df1d3ff8.slice/crio-760d692c33d6575a11f554ee5f5a1f061a466716f1708148a38882e0b42e6e20 WatchSource:0}: Error finding container 760d692c33d6575a11f554ee5f5a1f061a466716f1708148a38882e0b42e6e20: Status 404 returned error can't find the container with id 760d692c33d6575a11f554ee5f5a1f061a466716f1708148a38882e0b42e6e20
Jul 26 18:48:45.147970 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.147864    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm" event={"ID":"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85","Type":"ContainerStarted","Data":"7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03"}
Jul 26 18:48:45.148138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.148033    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:45.148138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.148072    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm" event={"ID":"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85","Type":"ContainerStarted","Data":"e43c71bc22e5b33312ed67c1ae5ae83a79eff3a44b4811a71e43a997e0737a39"}
Jul 26 18:48:45.149397 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.149372    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm" event={"ID":"2df467ea-0597-4792-bf7d-5b60df1d3ff8","Type":"ContainerStarted","Data":"266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10"}
Jul 26 18:48:45.149494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.149419    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm" event={"ID":"2df467ea-0597-4792-bf7d-5b60df1d3ff8","Type":"ContainerStarted","Data":"760d692c33d6575a11f554ee5f5a1f061a466716f1708148a38882e0b42e6e20"}
Jul 26 18:48:45.149494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.149442    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:45.163789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.163736    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm" podStartSLOduration=1.1637187230000001 podStartE2EDuration="1.163718723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:48:45.163136237 +0000 UTC m=+3079.353165988" watchObservedRunningTime="2026-07-26 18:48:45.163718723 +0000 UTC m=+3079.353748479"
Jul 26 18:48:45.182558 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:45.182498    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm" podStartSLOduration=1.18247397 podStartE2EDuration="1.18247397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:48:45.180808074 +0000 UTC m=+3079.370837830" watchObservedRunningTime="2026-07-26 18:48:45.18247397 +0000 UTC m=+3079.372503731"
Jul 26 18:48:47.621541 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:47.621500    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:48:56.157403 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:56.157373    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:48:56.157809 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:56.157453    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:48:58.887891 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:58.887853    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"]
Jul 26 18:48:58.891551 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:58.891532    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:48:58.894174 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:58.894151    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:48:58.894285 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:58.894158    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:48:58.898609 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:58.898586    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"]
Jul 26 18:48:58.909622 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:58.909593    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c058973-60e9-4850-a60a-a4c6132f8d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff\" (UID: \"4c058973-60e9-4850-a60a-a4c6132f8d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:48:59.010858 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:59.010820    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c058973-60e9-4850-a60a-a4c6132f8d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff\" (UID: \"4c058973-60e9-4850-a60a-a4c6132f8d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:48:59.011465 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:59.011443    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c058973-60e9-4850-a60a-a4c6132f8d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff\" (UID: \"4c058973-60e9-4850-a60a-a4c6132f8d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:48:59.204109 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:59.204016    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:48:59.338146 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:48:59.338120    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"]
Jul 26 18:48:59.340289 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:48:59.340259    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c058973_60e9_4850_a60a_a4c6132f8d15.slice/crio-88378c200a9b596fd2560a75f9bb81f475bee16064aa2e532f4b984a39a773b8 WatchSource:0}: Error finding container 88378c200a9b596fd2560a75f9bb81f475bee16064aa2e532f4b984a39a773b8: Status 404 returned error can't find the container with id 88378c200a9b596fd2560a75f9bb81f475bee16064aa2e532f4b984a39a773b8
Jul 26 18:49:00.231333 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:00.231289    2565 generic.go:356] "Generic (PLEG): container finished" podID="4c058973-60e9-4850-a60a-a4c6132f8d15" containerID="eec3b7dfbcfb38937e524fc97ba474e3b3f2c2cad44d38acb6731f593d5fc80f" exitCode=0
Jul 26 18:49:00.231745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:00.231339    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff" event={"ID":"4c058973-60e9-4850-a60a-a4c6132f8d15","Type":"ContainerDied","Data":"eec3b7dfbcfb38937e524fc97ba474e3b3f2c2cad44d38acb6731f593d5fc80f"}
Jul 26 18:49:00.231745 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:00.231393    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff" event={"ID":"4c058973-60e9-4850-a60a-a4c6132f8d15","Type":"ContainerStarted","Data":"88378c200a9b596fd2560a75f9bb81f475bee16064aa2e532f4b984a39a773b8"}
Jul 26 18:49:01.371184 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:01.371162    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:49:01.427811 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:01.427774    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c058973-60e9-4850-a60a-a4c6132f8d15-maas-api-service-ca\") pod \"4c058973-60e9-4850-a60a-a4c6132f8d15\" (UID: \"4c058973-60e9-4850-a60a-a4c6132f8d15\") "
Jul 26 18:49:01.428135 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:01.428112    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c058973-60e9-4850-a60a-a4c6132f8d15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c058973-60e9-4850-a60a-a4c6132f8d15" (UID: "4c058973-60e9-4850-a60a-a4c6132f8d15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:01.529071 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:01.528980    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c058973-60e9-4850-a60a-a4c6132f8d15-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:02.243278 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:02.243252    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"
Jul 26 18:49:02.243480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:02.243286    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff" event={"ID":"4c058973-60e9-4850-a60a-a4c6132f8d15","Type":"ContainerDied","Data":"88378c200a9b596fd2560a75f9bb81f475bee16064aa2e532f4b984a39a773b8"}
Jul 26 18:49:02.243480 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:02.243338    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="88378c200a9b596fd2560a75f9bb81f475bee16064aa2e532f4b984a39a773b8"
Jul 26 18:49:09.356169 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.356134    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"]
Jul 26 18:49:09.356963 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.356923    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm" podUID="a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" containerName="payload-processing" containerID="cri-o://7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03" gracePeriod=30
Jul 26 18:49:09.364434 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.364373    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"]
Jul 26 18:49:09.366335 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.366277    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm" podUID="2df467ea-0597-4792-bf7d-5b60df1d3ff8" containerName="payload-pre-processing" containerID="cri-o://266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10" gracePeriod=30
Jul 26 18:49:09.640326 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.640296    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:49:09.683789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.683765    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-plugins-config-volume\") pod \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") "
Jul 26 18:49:09.683951 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.683853    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8rr7\" (UniqueName: \"kubernetes.io/projected/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-kube-api-access-g8rr7\") pod \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\" (UID: \"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85\") "
Jul 26 18:49:09.684196 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.684169    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" (UID: "a7e28df0-e5dd-4c8b-a2aa-fd563e206b85"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:09.685913 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.685889    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-kube-api-access-g8rr7" (OuterVolumeSpecName: "kube-api-access-g8rr7") pod "a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" (UID: "a7e28df0-e5dd-4c8b-a2aa-fd563e206b85"). InnerVolumeSpecName "kube-api-access-g8rr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:09.686661 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.686640    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:49:09.784309 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.784278    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df467ea-0597-4792-bf7d-5b60df1d3ff8-plugins-config-volume\") pod \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") "
Jul 26 18:49:09.784494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.784383    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psdf7\" (UniqueName: \"kubernetes.io/projected/2df467ea-0597-4792-bf7d-5b60df1d3ff8-kube-api-access-psdf7\") pod \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\" (UID: \"2df467ea-0597-4792-bf7d-5b60df1d3ff8\") "
Jul 26 18:49:09.784538 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.784503    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8rr7\" (UniqueName: \"kubernetes.io/projected/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-kube-api-access-g8rr7\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:09.784538 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.784513    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:09.784640 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.784618    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2df467ea-0597-4792-bf7d-5b60df1d3ff8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2df467ea-0597-4792-bf7d-5b60df1d3ff8" (UID: "2df467ea-0597-4792-bf7d-5b60df1d3ff8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:09.786369 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.786342    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2df467ea-0597-4792-bf7d-5b60df1d3ff8-kube-api-access-psdf7" (OuterVolumeSpecName: "kube-api-access-psdf7") pod "2df467ea-0597-4792-bf7d-5b60df1d3ff8" (UID: "2df467ea-0597-4792-bf7d-5b60df1d3ff8"). InnerVolumeSpecName "kube-api-access-psdf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:09.885686 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.885607    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psdf7\" (UniqueName: \"kubernetes.io/projected/2df467ea-0597-4792-bf7d-5b60df1d3ff8-kube-api-access-psdf7\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:09.885686 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:09.885635    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2df467ea-0597-4792-bf7d-5b60df1d3ff8-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:10.285278 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.285241    2565 generic.go:356] "Generic (PLEG): container finished" podID="a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" containerID="7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03" exitCode=0
Jul 26 18:49:10.285481 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.285297    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"
Jul 26 18:49:10.285481 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.285334    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm" event={"ID":"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85","Type":"ContainerDied","Data":"7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03"}
Jul 26 18:49:10.285481 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.285394    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm" event={"ID":"a7e28df0-e5dd-4c8b-a2aa-fd563e206b85","Type":"ContainerDied","Data":"e43c71bc22e5b33312ed67c1ae5ae83a79eff3a44b4811a71e43a997e0737a39"}
Jul 26 18:49:10.285481 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.285429    2565 scope.go:117] "RemoveContainer" containerID="7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03"
Jul 26 18:49:10.286690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.286669    2565 generic.go:356] "Generic (PLEG): container finished" podID="2df467ea-0597-4792-bf7d-5b60df1d3ff8" containerID="266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10" exitCode=0
Jul 26 18:49:10.286810 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.286737    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"
Jul 26 18:49:10.286810 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.286735    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm" event={"ID":"2df467ea-0597-4792-bf7d-5b60df1d3ff8","Type":"ContainerDied","Data":"266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10"}
Jul 26 18:49:10.286912 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.286847    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm" event={"ID":"2df467ea-0597-4792-bf7d-5b60df1d3ff8","Type":"ContainerDied","Data":"760d692c33d6575a11f554ee5f5a1f061a466716f1708148a38882e0b42e6e20"}
Jul 26 18:49:10.296718 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.296680    2565 scope.go:117] "RemoveContainer" containerID="7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03"
Jul 26 18:49:10.297152 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:49:10.297131    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03\": container with ID starting with 7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03 not found: ID does not exist" containerID="7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03"
Jul 26 18:49:10.297248 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.297161    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03"} err="failed to get container status \"7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03\": rpc error: code = NotFound desc = could not find container \"7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03\": container with ID starting with 7c9d31573b304b108789fb0b6812f3d5693786ecb20c1b79a6bd0f9a061f5e03 not found: ID does not exist"
Jul 26 18:49:10.297248 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.297180    2565 scope.go:117] "RemoveContainer" containerID="266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10"
Jul 26 18:49:10.305668 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.305652    2565 scope.go:117] "RemoveContainer" containerID="266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10"
Jul 26 18:49:10.305928 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:49:10.305910    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10\": container with ID starting with 266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10 not found: ID does not exist" containerID="266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10"
Jul 26 18:49:10.305991 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.305935    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10"} err="failed to get container status \"266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10\": rpc error: code = NotFound desc = could not find container \"266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10\": container with ID starting with 266284cf4bf43b51edc1fcae103c02a4f76be55d28b38f6c94d60c47edbeaa10 not found: ID does not exist"
Jul 26 18:49:10.320429 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.320403    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"]
Jul 26 18:49:10.320549 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.320436    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-786276-cbd69b448-d65nm"]
Jul 26 18:49:10.328772 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.328747    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"]
Jul 26 18:49:10.333054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:10.333029    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-786276-5549798b68-2d2gm"]
Jul 26 18:49:12.318939 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:12.318908    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2df467ea-0597-4792-bf7d-5b60df1d3ff8" path="/var/lib/kubelet/pods/2df467ea-0597-4792-bf7d-5b60df1d3ff8/volumes"
Jul 26 18:49:12.319406 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:12.319269    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" path="/var/lib/kubelet/pods/a7e28df0-e5dd-4c8b-a2aa-fd563e206b85/volumes"
Jul 26 18:49:13.970198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:13.970166    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"]
Jul 26 18:49:13.972267 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:13.972246    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-786276-kwsff"]
Jul 26 18:49:14.319068 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:14.318973    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c058973-60e9-4850-a60a-a4c6132f8d15" path="/var/lib/kubelet/pods/4c058973-60e9-4850-a60a-a4c6132f8d15/volumes"
Jul 26 18:49:15.018321 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:15.018289    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:49:23.943135 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943096    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"]
Jul 26 18:49:23.943533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943447    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c058973-60e9-4850-a60a-a4c6132f8d15" containerName="revoke-keys"
Jul 26 18:49:23.943533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943459    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c058973-60e9-4850-a60a-a4c6132f8d15" containerName="revoke-keys"
Jul 26 18:49:23.943533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943474    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2df467ea-0597-4792-bf7d-5b60df1d3ff8" containerName="payload-pre-processing"
Jul 26 18:49:23.943533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943479    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2df467ea-0597-4792-bf7d-5b60df1d3ff8" containerName="payload-pre-processing"
Jul 26 18:49:23.943533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943489    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" containerName="payload-processing"
Jul 26 18:49:23.943533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943495    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" containerName="payload-processing"
Jul 26 18:49:23.943798 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943560    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2df467ea-0597-4792-bf7d-5b60df1d3ff8" containerName="payload-pre-processing"
Jul 26 18:49:23.943798 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943567    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c058973-60e9-4850-a60a-a4c6132f8d15" containerName="revoke-keys"
Jul 26 18:49:23.943798 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.943574    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e28df0-e5dd-4c8b-a2aa-fd563e206b85" containerName="payload-processing"
Jul 26 18:49:23.947936 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.947919    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:23.951182 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.951156    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:49:23.951354 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.951156    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:49:23.953401 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:23.953374    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"]
Jul 26 18:49:24.004337 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:24.004299    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66266824-4fe1-4b8b-8008-3f882cc37d0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-33964c-h688q\" (UID: \"66266824-4fe1-4b8b-8008-3f882cc37d0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:24.105033 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:24.104985    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66266824-4fe1-4b8b-8008-3f882cc37d0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-33964c-h688q\" (UID: \"66266824-4fe1-4b8b-8008-3f882cc37d0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:24.105638 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:24.105616    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66266824-4fe1-4b8b-8008-3f882cc37d0b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-33964c-h688q\" (UID: \"66266824-4fe1-4b8b-8008-3f882cc37d0b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:24.259632 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:24.259599    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:24.397400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:24.397377    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"]
Jul 26 18:49:24.399039 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:49:24.399011    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66266824_4fe1_4b8b_8008_3f882cc37d0b.slice/crio-252030e61e79b073807eeff0c5816b058e0f71aefae50a2079dfb032635661e9 WatchSource:0}: Error finding container 252030e61e79b073807eeff0c5816b058e0f71aefae50a2079dfb032635661e9: Status 404 returned error can't find the container with id 252030e61e79b073807eeff0c5816b058e0f71aefae50a2079dfb032635661e9
Jul 26 18:49:25.370207 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:25.370116    2565 generic.go:356] "Generic (PLEG): container finished" podID="66266824-4fe1-4b8b-8008-3f882cc37d0b" containerID="f36470269c5d56df0886d8bacd370c7f8587a14169c967bd932fde28288b751c" exitCode=0
Jul 26 18:49:25.370567 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:25.370199    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q" event={"ID":"66266824-4fe1-4b8b-8008-3f882cc37d0b","Type":"ContainerDied","Data":"f36470269c5d56df0886d8bacd370c7f8587a14169c967bd932fde28288b751c"}
Jul 26 18:49:25.370567 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:25.370249    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q" event={"ID":"66266824-4fe1-4b8b-8008-3f882cc37d0b","Type":"ContainerStarted","Data":"252030e61e79b073807eeff0c5816b058e0f71aefae50a2079dfb032635661e9"}
Jul 26 18:49:26.512167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:26.512141    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:26.624537 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:26.624505    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66266824-4fe1-4b8b-8008-3f882cc37d0b-maas-api-service-ca\") pod \"66266824-4fe1-4b8b-8008-3f882cc37d0b\" (UID: \"66266824-4fe1-4b8b-8008-3f882cc37d0b\") "
Jul 26 18:49:26.624880 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:26.624856    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66266824-4fe1-4b8b-8008-3f882cc37d0b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66266824-4fe1-4b8b-8008-3f882cc37d0b" (UID: "66266824-4fe1-4b8b-8008-3f882cc37d0b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:26.725377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:26.725350    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66266824-4fe1-4b8b-8008-3f882cc37d0b-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:27.382087 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:27.382058    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"
Jul 26 18:49:27.382270 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:27.382088    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q" event={"ID":"66266824-4fe1-4b8b-8008-3f882cc37d0b","Type":"ContainerDied","Data":"252030e61e79b073807eeff0c5816b058e0f71aefae50a2079dfb032635661e9"}
Jul 26 18:49:27.382270 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:27.382133    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="252030e61e79b073807eeff0c5816b058e0f71aefae50a2079dfb032635661e9"
Jul 26 18:49:38.619411 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:38.616609    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:49:39.411746 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.411710    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"]
Jul 26 18:49:39.417560 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.417524    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"]
Jul 26 18:49:39.417852 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.417801    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62" podUID="bb01b4f6-3bdc-4843-83c9-d5833cbec96d" containerName="payload-pre-processing" containerID="cri-o://c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf" gracePeriod=30
Jul 26 18:49:39.448565 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.448529    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr" podUID="e0e0d3c4-020d-4a0b-81d1-4a77a7371662" containerName="payload-processing" containerID="cri-o://83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6" gracePeriod=30
Jul 26 18:49:39.724163 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.724131    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:49:39.726945 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.726926    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:49:39.842878 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.842838    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-plugins-config-volume\") pod \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") "
Jul 26 18:49:39.843061 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.842891    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-plugins-config-volume\") pod \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") "
Jul 26 18:49:39.843061 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.842915    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djl6k\" (UniqueName: \"kubernetes.io/projected/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-kube-api-access-djl6k\") pod \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\" (UID: \"e0e0d3c4-020d-4a0b-81d1-4a77a7371662\") "
Jul 26 18:49:39.843061 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.842930    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tsrb\" (UniqueName: \"kubernetes.io/projected/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-kube-api-access-8tsrb\") pod \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\" (UID: \"bb01b4f6-3bdc-4843-83c9-d5833cbec96d\") "
Jul 26 18:49:39.843347 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.843315    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0e0d3c4-020d-4a0b-81d1-4a77a7371662" (UID: "e0e0d3c4-020d-4a0b-81d1-4a77a7371662"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:39.843410 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.843316    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bb01b4f6-3bdc-4843-83c9-d5833cbec96d" (UID: "bb01b4f6-3bdc-4843-83c9-d5833cbec96d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:39.845042 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.845019    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-kube-api-access-djl6k" (OuterVolumeSpecName: "kube-api-access-djl6k") pod "e0e0d3c4-020d-4a0b-81d1-4a77a7371662" (UID: "e0e0d3c4-020d-4a0b-81d1-4a77a7371662"). InnerVolumeSpecName "kube-api-access-djl6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:39.845139 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.845067    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-kube-api-access-8tsrb" (OuterVolumeSpecName: "kube-api-access-8tsrb") pod "bb01b4f6-3bdc-4843-83c9-d5833cbec96d" (UID: "bb01b4f6-3bdc-4843-83c9-d5833cbec96d"). InnerVolumeSpecName "kube-api-access-8tsrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:39.943623 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.943587    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:39.943623 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.943617    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:39.943623 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.943628    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djl6k\" (UniqueName: \"kubernetes.io/projected/e0e0d3c4-020d-4a0b-81d1-4a77a7371662-kube-api-access-djl6k\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:39.943870 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:39.943637    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tsrb\" (UniqueName: \"kubernetes.io/projected/bb01b4f6-3bdc-4843-83c9-d5833cbec96d-kube-api-access-8tsrb\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:49:40.454184 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.454148    2565 generic.go:356] "Generic (PLEG): container finished" podID="e0e0d3c4-020d-4a0b-81d1-4a77a7371662" containerID="83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6" exitCode=0
Jul 26 18:49:40.454377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.454197    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"
Jul 26 18:49:40.454377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.454237    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr" event={"ID":"e0e0d3c4-020d-4a0b-81d1-4a77a7371662","Type":"ContainerDied","Data":"83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6"}
Jul 26 18:49:40.454377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.454301    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr" event={"ID":"e0e0d3c4-020d-4a0b-81d1-4a77a7371662","Type":"ContainerDied","Data":"230af2511e9c6144e19e70de41c708d32bec7e64bf424c0caf9ebd842ee2616c"}
Jul 26 18:49:40.454377 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.454333    2565 scope.go:117] "RemoveContainer" containerID="83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6"
Jul 26 18:49:40.455716 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.455673    2565 generic.go:356] "Generic (PLEG): container finished" podID="bb01b4f6-3bdc-4843-83c9-d5833cbec96d" containerID="c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf" exitCode=0
Jul 26 18:49:40.455785 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.455729    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"
Jul 26 18:49:40.455785 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.455759    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62" event={"ID":"bb01b4f6-3bdc-4843-83c9-d5833cbec96d","Type":"ContainerDied","Data":"c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf"}
Jul 26 18:49:40.455862 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.455801    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62" event={"ID":"bb01b4f6-3bdc-4843-83c9-d5833cbec96d","Type":"ContainerDied","Data":"5344f0fa873898d20cd2a66beabd1d60c6d84fb255d5a48136d1f6a813a81802"}
Jul 26 18:49:40.463414 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.463398    2565 scope.go:117] "RemoveContainer" containerID="83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6"
Jul 26 18:49:40.463662 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:49:40.463640    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6\": container with ID starting with 83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6 not found: ID does not exist" containerID="83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6"
Jul 26 18:49:40.463780 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.463675    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6"} err="failed to get container status \"83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6\": rpc error: code = NotFound desc = could not find container \"83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6\": container with ID starting with 83f320dd7a2acc9bfd3a851cfff28512e91e6aac711928516be1425fdb3513f6 not found: ID does not exist"
Jul 26 18:49:40.463780 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.463711    2565 scope.go:117] "RemoveContainer" containerID="c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf"
Jul 26 18:49:40.472379 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.472348    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"]
Jul 26 18:49:40.473681 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.473664    2565 scope.go:117] "RemoveContainer" containerID="c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf"
Jul 26 18:49:40.474001 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:49:40.473981    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf\": container with ID starting with c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf not found: ID does not exist" containerID="c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf"
Jul 26 18:49:40.474088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.474014    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf"} err="failed to get container status \"c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf\": rpc error: code = NotFound desc = could not find container \"c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf\": container with ID starting with c7cc7fa53a5e63dd4b371ea4680406de121a0c9a154799ce80cdca6e83329fdf not found: ID does not exist"
Jul 26 18:49:40.475789 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.475769    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-33964c-7c9987bbfd-j8xtr"]
Jul 26 18:49:40.491100 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.491078    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"]
Jul 26 18:49:40.495053 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:40.495033    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-33964c-66b6df5dd8-f8v62"]
Jul 26 18:49:42.320053 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:42.320015    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb01b4f6-3bdc-4843-83c9-d5833cbec96d" path="/var/lib/kubelet/pods/bb01b4f6-3bdc-4843-83c9-d5833cbec96d/volumes"
Jul 26 18:49:42.320555 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:42.320513    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0e0d3c4-020d-4a0b-81d1-4a77a7371662" path="/var/lib/kubelet/pods/e0e0d3c4-020d-4a0b-81d1-4a77a7371662/volumes"
Jul 26 18:49:44.019312 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:44.019271    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"]
Jul 26 18:49:44.022292 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:44.022267    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-33964c-h688q"]
Jul 26 18:49:44.318976 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:44.318895    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66266824-4fe1-4b8b-8008-3f882cc37d0b" path="/var/lib/kubelet/pods/66266824-4fe1-4b8b-8008-3f882cc37d0b/volumes"
Jul 26 18:49:49.110674 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:49.110591    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:49:49.376909 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:49:49.376810    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"]
Jul 26 18:50:04.395783 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.395735    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2" podUID="144c575f-5f78-48eb-8dfe-159d23d5fc56" containerName="main" containerID="cri-o://ef88440edb1bb738a50c3f6cf709d31d612731ffdb0f3957036e018d04496e00" gracePeriod=45
Jul 26 18:50:04.582878 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.582845    2565 generic.go:356] "Generic (PLEG): container finished" podID="144c575f-5f78-48eb-8dfe-159d23d5fc56" containerID="ef88440edb1bb738a50c3f6cf709d31d612731ffdb0f3957036e018d04496e00" exitCode=0
Jul 26 18:50:04.583051 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.582871    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2" event={"ID":"144c575f-5f78-48eb-8dfe-159d23d5fc56","Type":"ContainerDied","Data":"ef88440edb1bb738a50c3f6cf709d31d612731ffdb0f3957036e018d04496e00"}
Jul 26 18:50:04.650489 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.650422    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:50:04.666455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666410    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-dshm\") pod \"144c575f-5f78-48eb-8dfe-159d23d5fc56\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") "
Jul 26 18:50:04.666632 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666464    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg8lk\" (UniqueName: \"kubernetes.io/projected/144c575f-5f78-48eb-8dfe-159d23d5fc56-kube-api-access-kg8lk\") pod \"144c575f-5f78-48eb-8dfe-159d23d5fc56\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") "
Jul 26 18:50:04.666632 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666513    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-home\") pod \"144c575f-5f78-48eb-8dfe-159d23d5fc56\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") "
Jul 26 18:50:04.666632 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666542    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144c575f-5f78-48eb-8dfe-159d23d5fc56-tls-certs\") pod \"144c575f-5f78-48eb-8dfe-159d23d5fc56\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") "
Jul 26 18:50:04.666632 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666578    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-tmp-dir\") pod \"144c575f-5f78-48eb-8dfe-159d23d5fc56\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") "
Jul 26 18:50:04.666892 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666644    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-model-cache\") pod \"144c575f-5f78-48eb-8dfe-159d23d5fc56\" (UID: \"144c575f-5f78-48eb-8dfe-159d23d5fc56\") "
Jul 26 18:50:04.666954 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666893    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-home" (OuterVolumeSpecName: "home") pod "144c575f-5f78-48eb-8dfe-159d23d5fc56" (UID: "144c575f-5f78-48eb-8dfe-159d23d5fc56"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:04.667284 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.666995    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "144c575f-5f78-48eb-8dfe-159d23d5fc56" (UID: "144c575f-5f78-48eb-8dfe-159d23d5fc56"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:04.667284 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.667036    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-model-cache" (OuterVolumeSpecName: "model-cache") pod "144c575f-5f78-48eb-8dfe-159d23d5fc56" (UID: "144c575f-5f78-48eb-8dfe-159d23d5fc56"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:04.669425 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.669377    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/144c575f-5f78-48eb-8dfe-159d23d5fc56-kube-api-access-kg8lk" (OuterVolumeSpecName: "kube-api-access-kg8lk") pod "144c575f-5f78-48eb-8dfe-159d23d5fc56" (UID: "144c575f-5f78-48eb-8dfe-159d23d5fc56"). InnerVolumeSpecName "kube-api-access-kg8lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:04.669523 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.669453    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-dshm" (OuterVolumeSpecName: "dshm") pod "144c575f-5f78-48eb-8dfe-159d23d5fc56" (UID: "144c575f-5f78-48eb-8dfe-159d23d5fc56"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:50:04.669523 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.669503    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/144c575f-5f78-48eb-8dfe-159d23d5fc56-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "144c575f-5f78-48eb-8dfe-159d23d5fc56" (UID: "144c575f-5f78-48eb-8dfe-159d23d5fc56"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:50:04.767629 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.767589    2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-dshm\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:04.767629 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.767621    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg8lk\" (UniqueName: \"kubernetes.io/projected/144c575f-5f78-48eb-8dfe-159d23d5fc56-kube-api-access-kg8lk\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:04.767629 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.767632    2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-home\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:04.767629 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.767640    2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/144c575f-5f78-48eb-8dfe-159d23d5fc56-tls-certs\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:04.767943 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.767650    2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-tmp-dir\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:04.767943 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:04.767657    2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/144c575f-5f78-48eb-8dfe-159d23d5fc56-model-cache\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:05.589780 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:05.589752    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2" event={"ID":"144c575f-5f78-48eb-8dfe-159d23d5fc56","Type":"ContainerDied","Data":"933032ff4e9bf17065ce3ca03bc291f41f5dae4e782496f52a0de6109c1aa7d2"}
Jul 26 18:50:05.589780 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:05.589767    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"
Jul 26 18:50:05.590297 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:05.589801    2565 scope.go:117] "RemoveContainer" containerID="ef88440edb1bb738a50c3f6cf709d31d612731ffdb0f3957036e018d04496e00"
Jul 26 18:50:05.611615 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:05.611583    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"]
Jul 26 18:50:05.618161 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:05.618135    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-33964c/ipp-route-33964c-kserve-bbf57d48c-lhrv2"]
Jul 26 18:50:06.318816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:06.318784    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="144c575f-5f78-48eb-8dfe-159d23d5fc56" path="/var/lib/kubelet/pods/144c575f-5f78-48eb-8dfe-159d23d5fc56/volumes"
Jul 26 18:50:32.959563 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959526    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"]
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959889    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66266824-4fe1-4b8b-8008-3f882cc37d0b" containerName="revoke-keys"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959902    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="66266824-4fe1-4b8b-8008-3f882cc37d0b" containerName="revoke-keys"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959918    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb01b4f6-3bdc-4843-83c9-d5833cbec96d" containerName="payload-pre-processing"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959923    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb01b4f6-3bdc-4843-83c9-d5833cbec96d" containerName="payload-pre-processing"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959929    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e0d3c4-020d-4a0b-81d1-4a77a7371662" containerName="payload-processing"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959936    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e0d3c4-020d-4a0b-81d1-4a77a7371662" containerName="payload-processing"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959943    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="144c575f-5f78-48eb-8dfe-159d23d5fc56" containerName="main"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.959948    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="144c575f-5f78-48eb-8dfe-159d23d5fc56" containerName="main"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.960000    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e0d3c4-020d-4a0b-81d1-4a77a7371662" containerName="payload-processing"
Jul 26 18:50:32.960007 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.960006    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="144c575f-5f78-48eb-8dfe-159d23d5fc56" containerName="main"
Jul 26 18:50:32.960312 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.960014    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="66266824-4fe1-4b8b-8008-3f882cc37d0b" containerName="revoke-keys"
Jul 26 18:50:32.960312 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.960021    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb01b4f6-3bdc-4843-83c9-d5833cbec96d" containerName="payload-pre-processing"
Jul 26 18:50:32.964387 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.964366    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:32.967000 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.966967    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:50:32.967000 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.966983    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:50:32.971330 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:32.971298    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"]
Jul 26 18:50:33.110343 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.110305    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3862fdc-1bac-4ab5-9a70-1540768818b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b\" (UID: \"a3862fdc-1bac-4ab5-9a70-1540768818b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:33.211170 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.211073    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3862fdc-1bac-4ab5-9a70-1540768818b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b\" (UID: \"a3862fdc-1bac-4ab5-9a70-1540768818b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:33.211736 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.211689    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3862fdc-1bac-4ab5-9a70-1540768818b2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b\" (UID: \"a3862fdc-1bac-4ab5-9a70-1540768818b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:33.289202 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.289165    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:33.422124 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.422101    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"]
Jul 26 18:50:33.423991 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:50:33.423963    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3862fdc_1bac_4ab5_9a70_1540768818b2.slice/crio-fcc58cbf729a0be575785d46e38dc0ed389d3e3a8a595cde31ef8ffd57fd2990 WatchSource:0}: Error finding container fcc58cbf729a0be575785d46e38dc0ed389d3e3a8a595cde31ef8ffd57fd2990: Status 404 returned error can't find the container with id fcc58cbf729a0be575785d46e38dc0ed389d3e3a8a595cde31ef8ffd57fd2990
Jul 26 18:50:33.750917 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.750879    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b" event={"ID":"a3862fdc-1bac-4ab5-9a70-1540768818b2","Type":"ContainerStarted","Data":"a667fe71a2220139966bac7d3871acfb9305d24acfad3f7da7bac62456094ea9"}
Jul 26 18:50:33.751097 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.750941    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b" event={"ID":"a3862fdc-1bac-4ab5-9a70-1540768818b2","Type":"ContainerStarted","Data":"fcc58cbf729a0be575785d46e38dc0ed389d3e3a8a595cde31ef8ffd57fd2990"}
Jul 26 18:50:33.768670 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:33.768418    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b" podStartSLOduration=1.7683995750000001 podStartE2EDuration="1.768399575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:33.768198022 +0000 UTC m=+3187.958227779" watchObservedRunningTime="2026-07-26 18:50:33.768399575 +0000 UTC m=+3187.958429332"
Jul 26 18:50:34.757601 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:34.757561    2565 generic.go:356] "Generic (PLEG): container finished" podID="a3862fdc-1bac-4ab5-9a70-1540768818b2" containerID="a667fe71a2220139966bac7d3871acfb9305d24acfad3f7da7bac62456094ea9" exitCode=0
Jul 26 18:50:34.758036 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:34.757616    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b" event={"ID":"a3862fdc-1bac-4ab5-9a70-1540768818b2","Type":"ContainerDied","Data":"a667fe71a2220139966bac7d3871acfb9305d24acfad3f7da7bac62456094ea9"}
Jul 26 18:50:35.894467 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:35.894441    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:36.040280 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:36.040194    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3862fdc-1bac-4ab5-9a70-1540768818b2-maas-api-service-ca\") pod \"a3862fdc-1bac-4ab5-9a70-1540768818b2\" (UID: \"a3862fdc-1bac-4ab5-9a70-1540768818b2\") "
Jul 26 18:50:36.040551 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:36.040530    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3862fdc-1bac-4ab5-9a70-1540768818b2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a3862fdc-1bac-4ab5-9a70-1540768818b2" (UID: "a3862fdc-1bac-4ab5-9a70-1540768818b2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:36.141786 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:36.141751    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3862fdc-1bac-4ab5-9a70-1540768818b2-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:36.768627 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:36.768585    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b" event={"ID":"a3862fdc-1bac-4ab5-9a70-1540768818b2","Type":"ContainerDied","Data":"fcc58cbf729a0be575785d46e38dc0ed389d3e3a8a595cde31ef8ffd57fd2990"}
Jul 26 18:50:36.768822 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:36.768639    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcc58cbf729a0be575785d46e38dc0ed389d3e3a8a595cde31ef8ffd57fd2990"
Jul 26 18:50:36.768822 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:36.768603    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"
Jul 26 18:50:43.550993 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.550960    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"]
Jul 26 18:50:43.551508 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.551183    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw" podUID="3cb5975f-a6de-4a71-8c8e-435b68c0d26c" containerName="payload-processing" containerID="cri-o://b1512591157d89f03cdf33471ed549f803317a6cb131772d7e2fe6f391da0cd4" gracePeriod=30
Jul 26 18:50:43.568753 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.568716    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"]
Jul 26 18:50:43.569033 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.568967    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2" podUID="4f611942-fe45-4177-9447-5968b6e942c7" containerName="payload-pre-processing" containerID="cri-o://76d9c0a73988a7877f89f75e523cf34fa1e55143ecb7493df35444c3cedec2f7" gracePeriod=30
Jul 26 18:50:43.806998 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.806903    2565 generic.go:356] "Generic (PLEG): container finished" podID="3cb5975f-a6de-4a71-8c8e-435b68c0d26c" containerID="b1512591157d89f03cdf33471ed549f803317a6cb131772d7e2fe6f391da0cd4" exitCode=0
Jul 26 18:50:43.807159 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.807056    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw" event={"ID":"3cb5975f-a6de-4a71-8c8e-435b68c0d26c","Type":"ContainerDied","Data":"b1512591157d89f03cdf33471ed549f803317a6cb131772d7e2fe6f391da0cd4"}
Jul 26 18:50:43.808759 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.808732    2565 generic.go:356] "Generic (PLEG): container finished" podID="4f611942-fe45-4177-9447-5968b6e942c7" containerID="76d9c0a73988a7877f89f75e523cf34fa1e55143ecb7493df35444c3cedec2f7" exitCode=0
Jul 26 18:50:43.808890 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.808828    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2" event={"ID":"4f611942-fe45-4177-9447-5968b6e942c7","Type":"ContainerDied","Data":"76d9c0a73988a7877f89f75e523cf34fa1e55143ecb7493df35444c3cedec2f7"}
Jul 26 18:50:43.848994 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.848965    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:50:43.882237 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:43.882206    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:50:44.011314 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.011268    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f611942-fe45-4177-9447-5968b6e942c7-plugins-config-volume\") pod \"4f611942-fe45-4177-9447-5968b6e942c7\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") "
Jul 26 18:50:44.011494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.011339    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn5fg\" (UniqueName: \"kubernetes.io/projected/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-kube-api-access-gn5fg\") pod \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") "
Jul 26 18:50:44.011494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.011392    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-plugins-config-volume\") pod \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\" (UID: \"3cb5975f-a6de-4a71-8c8e-435b68c0d26c\") "
Jul 26 18:50:44.011494 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.011414    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5svl\" (UniqueName: \"kubernetes.io/projected/4f611942-fe45-4177-9447-5968b6e942c7-kube-api-access-h5svl\") pod \"4f611942-fe45-4177-9447-5968b6e942c7\" (UID: \"4f611942-fe45-4177-9447-5968b6e942c7\") "
Jul 26 18:50:44.011752 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.011691    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f611942-fe45-4177-9447-5968b6e942c7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4f611942-fe45-4177-9447-5968b6e942c7" (UID: "4f611942-fe45-4177-9447-5968b6e942c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:44.011835 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.011806    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cb5975f-a6de-4a71-8c8e-435b68c0d26c" (UID: "3cb5975f-a6de-4a71-8c8e-435b68c0d26c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:44.013533 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.013510    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-kube-api-access-gn5fg" (OuterVolumeSpecName: "kube-api-access-gn5fg") pod "3cb5975f-a6de-4a71-8c8e-435b68c0d26c" (UID: "3cb5975f-a6de-4a71-8c8e-435b68c0d26c"). InnerVolumeSpecName "kube-api-access-gn5fg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:44.013600 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.013518    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f611942-fe45-4177-9447-5968b6e942c7-kube-api-access-h5svl" (OuterVolumeSpecName: "kube-api-access-h5svl") pod "4f611942-fe45-4177-9447-5968b6e942c7" (UID: "4f611942-fe45-4177-9447-5968b6e942c7"). InnerVolumeSpecName "kube-api-access-h5svl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:44.112840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.112762    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f611942-fe45-4177-9447-5968b6e942c7-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:44.112840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.112789    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn5fg\" (UniqueName: \"kubernetes.io/projected/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-kube-api-access-gn5fg\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:44.112840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.112799    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cb5975f-a6de-4a71-8c8e-435b68c0d26c-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:44.112840 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.112808    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5svl\" (UniqueName: \"kubernetes.io/projected/4f611942-fe45-4177-9447-5968b6e942c7-kube-api-access-h5svl\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:44.327906 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:50:44.327878    2565 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f611942_fe45_4177_9447_5968b6e942c7.slice/crio-3b2b5afc88433370883bbee5a328f55dfcc24244dafae1d9968c43afe5e6107d\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f611942_fe45_4177_9447_5968b6e942c7.slice\": RecentStats: unable to find data in memory cache]"
Jul 26 18:50:44.815085 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.814998    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"
Jul 26 18:50:44.815085 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.815037    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2" event={"ID":"4f611942-fe45-4177-9447-5968b6e942c7","Type":"ContainerDied","Data":"3b2b5afc88433370883bbee5a328f55dfcc24244dafae1d9968c43afe5e6107d"}
Jul 26 18:50:44.815609 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.815094    2565 scope.go:117] "RemoveContainer" containerID="76d9c0a73988a7877f89f75e523cf34fa1e55143ecb7493df35444c3cedec2f7"
Jul 26 18:50:44.816540 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.816518    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw" event={"ID":"3cb5975f-a6de-4a71-8c8e-435b68c0d26c","Type":"ContainerDied","Data":"9b848d78147d777c39d3a9c27196b65136c848ec529cc514475e99acc81be5e5"}
Jul 26 18:50:44.816654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.816546    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"
Jul 26 18:50:44.824563 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.824542    2565 scope.go:117] "RemoveContainer" containerID="b1512591157d89f03cdf33471ed549f803317a6cb131772d7e2fe6f391da0cd4"
Jul 26 18:50:44.835543 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.835521    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"]
Jul 26 18:50:44.838932 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.838906    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f6bce0-5cbc547644-rtlvw"]
Jul 26 18:50:44.849899 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.849873    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"]
Jul 26 18:50:44.858469 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:44.855080    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-f6bce0-544f654b96-c45x2"]
Jul 26 18:50:46.319556 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:46.319523    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cb5975f-a6de-4a71-8c8e-435b68c0d26c" path="/var/lib/kubelet/pods/3cb5975f-a6de-4a71-8c8e-435b68c0d26c/volumes"
Jul 26 18:50:46.319959 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:46.319894    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f611942-fe45-4177-9447-5968b6e942c7" path="/var/lib/kubelet/pods/4f611942-fe45-4177-9447-5968b6e942c7/volumes"
Jul 26 18:50:46.607036 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:46.606956    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:50:48.028287 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:48.028247    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"]
Jul 26 18:50:48.031009 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:48.030987    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f6bce0-h4t8b"]
Jul 26 18:50:48.318126 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:48.318049    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3862fdc-1bac-4ab5-9a70-1540768818b2" path="/var/lib/kubelet/pods/a3862fdc-1bac-4ab5-9a70-1540768818b2/volumes"
Jul 26 18:50:54.853852 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.853817    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"]
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854132    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cb5975f-a6de-4a71-8c8e-435b68c0d26c" containerName="payload-processing"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854142    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cb5975f-a6de-4a71-8c8e-435b68c0d26c" containerName="payload-processing"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854154    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3862fdc-1bac-4ab5-9a70-1540768818b2" containerName="revoke-keys"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854159    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3862fdc-1bac-4ab5-9a70-1540768818b2" containerName="revoke-keys"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854180    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f611942-fe45-4177-9447-5968b6e942c7" containerName="payload-pre-processing"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854186    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f611942-fe45-4177-9447-5968b6e942c7" containerName="payload-pre-processing"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854232    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cb5975f-a6de-4a71-8c8e-435b68c0d26c" containerName="payload-processing"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854240    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3862fdc-1bac-4ab5-9a70-1540768818b2" containerName="revoke-keys"
Jul 26 18:50:54.854249 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.854248    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f611942-fe45-4177-9447-5968b6e942c7" containerName="payload-pre-processing"
Jul 26 18:50:54.858793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.858774    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:54.861444 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.861422    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:50:54.862192 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.862170    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:50:54.864246 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.864217    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"]
Jul 26 18:50:54.889796 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.889758    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce8e3ea-f482-4024-9318-2f95d5dc8435-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5\" (UID: \"2ce8e3ea-f482-4024-9318-2f95d5dc8435\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:54.990485 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.990446    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce8e3ea-f482-4024-9318-2f95d5dc8435-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5\" (UID: \"2ce8e3ea-f482-4024-9318-2f95d5dc8435\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:54.991108 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:54.991087    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce8e3ea-f482-4024-9318-2f95d5dc8435-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5\" (UID: \"2ce8e3ea-f482-4024-9318-2f95d5dc8435\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:55.171299 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:55.171180    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:55.306209 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:55.306162    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"]
Jul 26 18:50:55.308411 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:50:55.308380    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce8e3ea_f482_4024_9318_2f95d5dc8435.slice/crio-3927e6e0252dea0efd5dc733c134cd1934f0ceefbe2969436e3313514a158249 WatchSource:0}: Error finding container 3927e6e0252dea0efd5dc733c134cd1934f0ceefbe2969436e3313514a158249: Status 404 returned error can't find the container with id 3927e6e0252dea0efd5dc733c134cd1934f0ceefbe2969436e3313514a158249
Jul 26 18:50:55.877363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:55.877278    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5" event={"ID":"2ce8e3ea-f482-4024-9318-2f95d5dc8435","Type":"ContainerStarted","Data":"eb41dbb0fdae31f9517f3f08e4cfb624713918efa23e992170c4aabd85be8230"}
Jul 26 18:50:55.877363 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:55.877330    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5" event={"ID":"2ce8e3ea-f482-4024-9318-2f95d5dc8435","Type":"ContainerStarted","Data":"3927e6e0252dea0efd5dc733c134cd1934f0ceefbe2969436e3313514a158249"}
Jul 26 18:50:55.893856 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:55.893803    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5" podStartSLOduration=1.893788361 podStartE2EDuration="1.893788361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:55.891877443 +0000 UTC m=+3210.081907230" watchObservedRunningTime="2026-07-26 18:50:55.893788361 +0000 UTC m=+3210.083818115"
Jul 26 18:50:56.884180 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:56.884147    2565 generic.go:356] "Generic (PLEG): container finished" podID="2ce8e3ea-f482-4024-9318-2f95d5dc8435" containerID="eb41dbb0fdae31f9517f3f08e4cfb624713918efa23e992170c4aabd85be8230" exitCode=0
Jul 26 18:50:56.884596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:56.884230    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5" event={"ID":"2ce8e3ea-f482-4024-9318-2f95d5dc8435","Type":"ContainerDied","Data":"eb41dbb0fdae31f9517f3f08e4cfb624713918efa23e992170c4aabd85be8230"}
Jul 26 18:50:58.021654 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.021626    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:58.116940 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.116904    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce8e3ea-f482-4024-9318-2f95d5dc8435-maas-api-service-ca\") pod \"2ce8e3ea-f482-4024-9318-2f95d5dc8435\" (UID: \"2ce8e3ea-f482-4024-9318-2f95d5dc8435\") "
Jul 26 18:50:58.117278 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.117255    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce8e3ea-f482-4024-9318-2f95d5dc8435-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ce8e3ea-f482-4024-9318-2f95d5dc8435" (UID: "2ce8e3ea-f482-4024-9318-2f95d5dc8435"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:58.218342 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.218268    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ce8e3ea-f482-4024-9318-2f95d5dc8435-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:50:58.895607 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.895577    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"
Jul 26 18:50:58.895607 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.895589    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5" event={"ID":"2ce8e3ea-f482-4024-9318-2f95d5dc8435","Type":"ContainerDied","Data":"3927e6e0252dea0efd5dc733c134cd1934f0ceefbe2969436e3313514a158249"}
Jul 26 18:50:58.895850 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:50:58.895632    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3927e6e0252dea0efd5dc733c134cd1934f0ceefbe2969436e3313514a158249"
Jul 26 18:51:05.295206 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.295170    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"]
Jul 26 18:51:05.299669 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.299639    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-37e93d-29751525-qsvqs"]
Jul 26 18:51:05.326603 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.326572    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"]
Jul 26 18:51:05.326810 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.326779    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj" podUID="6ba4853c-f927-44e9-a29b-c684df532f6b" containerName="payload-processing" containerID="cri-o://f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf" gracePeriod=30
Jul 26 18:51:05.333828 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.333800    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"]
Jul 26 18:51:05.334117 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.334044    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988" podUID="11857142-854f-4cbb-a8fe-bbba3707ee3a" containerName="payload-pre-processing" containerID="cri-o://980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b" gracePeriod=30
Jul 26 18:51:05.586794 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.586768    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:51:05.611656 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.611630    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:51:05.677036 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.677004    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba4853c-f927-44e9-a29b-c684df532f6b-plugins-config-volume\") pod \"6ba4853c-f927-44e9-a29b-c684df532f6b\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") "
Jul 26 18:51:05.677036 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.677035    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8vfl\" (UniqueName: \"kubernetes.io/projected/11857142-854f-4cbb-a8fe-bbba3707ee3a-kube-api-access-w8vfl\") pod \"11857142-854f-4cbb-a8fe-bbba3707ee3a\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") "
Jul 26 18:51:05.677253 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.677054    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11857142-854f-4cbb-a8fe-bbba3707ee3a-plugins-config-volume\") pod \"11857142-854f-4cbb-a8fe-bbba3707ee3a\" (UID: \"11857142-854f-4cbb-a8fe-bbba3707ee3a\") "
Jul 26 18:51:05.677253 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.677101    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z947r\" (UniqueName: \"kubernetes.io/projected/6ba4853c-f927-44e9-a29b-c684df532f6b-kube-api-access-z947r\") pod \"6ba4853c-f927-44e9-a29b-c684df532f6b\" (UID: \"6ba4853c-f927-44e9-a29b-c684df532f6b\") "
Jul 26 18:51:05.677455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.677428    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11857142-854f-4cbb-a8fe-bbba3707ee3a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "11857142-854f-4cbb-a8fe-bbba3707ee3a" (UID: "11857142-854f-4cbb-a8fe-bbba3707ee3a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:05.677455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.677442    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ba4853c-f927-44e9-a29b-c684df532f6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ba4853c-f927-44e9-a29b-c684df532f6b" (UID: "6ba4853c-f927-44e9-a29b-c684df532f6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:05.679248 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.679225    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11857142-854f-4cbb-a8fe-bbba3707ee3a-kube-api-access-w8vfl" (OuterVolumeSpecName: "kube-api-access-w8vfl") pod "11857142-854f-4cbb-a8fe-bbba3707ee3a" (UID: "11857142-854f-4cbb-a8fe-bbba3707ee3a"). InnerVolumeSpecName "kube-api-access-w8vfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:05.679351 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.679256    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ba4853c-f927-44e9-a29b-c684df532f6b-kube-api-access-z947r" (OuterVolumeSpecName: "kube-api-access-z947r") pod "6ba4853c-f927-44e9-a29b-c684df532f6b" (UID: "6ba4853c-f927-44e9-a29b-c684df532f6b"). InnerVolumeSpecName "kube-api-access-z947r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:05.778599 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.778562    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ba4853c-f927-44e9-a29b-c684df532f6b-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.778599 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.778593    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8vfl\" (UniqueName: \"kubernetes.io/projected/11857142-854f-4cbb-a8fe-bbba3707ee3a-kube-api-access-w8vfl\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.778599 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.778603    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/11857142-854f-4cbb-a8fe-bbba3707ee3a-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.778862 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.778613    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z947r\" (UniqueName: \"kubernetes.io/projected/6ba4853c-f927-44e9-a29b-c684df532f6b-kube-api-access-z947r\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.933177 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.933080    2565 generic.go:356] "Generic (PLEG): container finished" podID="11857142-854f-4cbb-a8fe-bbba3707ee3a" containerID="980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b" exitCode=0
Jul 26 18:51:05.933177 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.933132    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"
Jul 26 18:51:05.933414 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.933176    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988" event={"ID":"11857142-854f-4cbb-a8fe-bbba3707ee3a","Type":"ContainerDied","Data":"980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b"}
Jul 26 18:51:05.933414 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.933266    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988" event={"ID":"11857142-854f-4cbb-a8fe-bbba3707ee3a","Type":"ContainerDied","Data":"b77986e8f687c13927072fd98b0b18434eda1b2e9c1ed5bc6b06997d057e5e2b"}
Jul 26 18:51:05.933414 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.933291    2565 scope.go:117] "RemoveContainer" containerID="980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b"
Jul 26 18:51:05.934552 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.934532    2565 generic.go:356] "Generic (PLEG): container finished" podID="6ba4853c-f927-44e9-a29b-c684df532f6b" containerID="f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf" exitCode=0
Jul 26 18:51:05.934675 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.934570    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"
Jul 26 18:51:05.934753 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.934666    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj" event={"ID":"6ba4853c-f927-44e9-a29b-c684df532f6b","Type":"ContainerDied","Data":"f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf"}
Jul 26 18:51:05.934753 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.934730    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj" event={"ID":"6ba4853c-f927-44e9-a29b-c684df532f6b","Type":"ContainerDied","Data":"fdc0e21fab08cabcb67aad6fd070bf0c106198f0b00f1fd902a23676fb7b5873"}
Jul 26 18:51:05.945736 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.944817    2565 scope.go:117] "RemoveContainer" containerID="980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b"
Jul 26 18:51:05.946169 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:51:05.946150    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b\": container with ID starting with 980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b not found: ID does not exist" containerID="980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b"
Jul 26 18:51:05.946234 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.946176    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b"} err="failed to get container status \"980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b\": rpc error: code = NotFound desc = could not find container \"980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b\": container with ID starting with 980d5a19cfa732f15dbca5b5277a360d19c792d067b3ffbaf549aba383b82c8b not found: ID does not exist"
Jul 26 18:51:05.946234 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.946192    2565 scope.go:117] "RemoveContainer" containerID="f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf"
Jul 26 18:51:05.954105 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.954082    2565 scope.go:117] "RemoveContainer" containerID="f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf"
Jul 26 18:51:05.954354 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:51:05.954335    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf\": container with ID starting with f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf not found: ID does not exist" containerID="f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf"
Jul 26 18:51:05.954401 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.954363    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf"} err="failed to get container status \"f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf\": rpc error: code = NotFound desc = could not find container \"f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf\": container with ID starting with f124776fcd927d3da65c29558a8f407ab6075935e1c206ee34318e71a8a014bf not found: ID does not exist"
Jul 26 18:51:05.961467 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.961443    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"]
Jul 26 18:51:05.965641 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.965623    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-37e93d-76f975ccf6-4j988"]
Jul 26 18:51:05.975956 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.975934    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"]
Jul 26 18:51:05.978167 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:05.978147    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-37e93d-79586f5b4b-qq4vj"]
Jul 26 18:51:06.319319 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:06.319290    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07431173-3c12-4f26-8df6-70ec473aabe4" path="/var/lib/kubelet/pods/07431173-3c12-4f26-8df6-70ec473aabe4/volumes"
Jul 26 18:51:06.319718 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:06.319640    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11857142-854f-4cbb-a8fe-bbba3707ee3a" path="/var/lib/kubelet/pods/11857142-854f-4cbb-a8fe-bbba3707ee3a/volumes"
Jul 26 18:51:06.320027 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:06.320013    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ba4853c-f927-44e9-a29b-c684df532f6b" path="/var/lib/kubelet/pods/6ba4853c-f927-44e9-a29b-c684df532f6b/volumes"
Jul 26 18:51:08.222724 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:08.222667    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:51:09.934451 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:09.934417    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"]
Jul 26 18:51:09.938816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:09.938792    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-37e93d-bhkg5"]
Jul 26 18:51:10.318820 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:10.318786    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ce8e3ea-f482-4024-9318-2f95d5dc8435" path="/var/lib/kubelet/pods/2ce8e3ea-f482-4024-9318-2f95d5dc8435/volumes"
Jul 26 18:51:16.946342 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946307    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"]
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946607    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11857142-854f-4cbb-a8fe-bbba3707ee3a" containerName="payload-pre-processing"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946621    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="11857142-854f-4cbb-a8fe-bbba3707ee3a" containerName="payload-pre-processing"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946636    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ce8e3ea-f482-4024-9318-2f95d5dc8435" containerName="revoke-keys"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946641    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ce8e3ea-f482-4024-9318-2f95d5dc8435" containerName="revoke-keys"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946651    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ba4853c-f927-44e9-a29b-c684df532f6b" containerName="payload-processing"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946657    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ba4853c-f927-44e9-a29b-c684df532f6b" containerName="payload-processing"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946732    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ba4853c-f927-44e9-a29b-c684df532f6b" containerName="payload-processing"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946739    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="11857142-854f-4cbb-a8fe-bbba3707ee3a" containerName="payload-pre-processing"
Jul 26 18:51:16.946793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.946746    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ce8e3ea-f482-4024-9318-2f95d5dc8435" containerName="revoke-keys"
Jul 26 18:51:16.949852 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.949836    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:16.952500 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.952485    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:51:16.952606 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.952501    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zbcqv\""
Jul 26 18:51:16.962429 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:16.962396    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"]
Jul 26 18:51:17.071350 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.071314    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c2facf-9ed2-4729-9679-48a3c9add3c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r\" (UID: \"70c2facf-9ed2-4729-9679-48a3c9add3c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:17.172273 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.172239    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c2facf-9ed2-4729-9679-48a3c9add3c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r\" (UID: \"70c2facf-9ed2-4729-9679-48a3c9add3c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:17.172896 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.172877    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c2facf-9ed2-4729-9679-48a3c9add3c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r\" (UID: \"70c2facf-9ed2-4729-9679-48a3c9add3c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:17.262800 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.262765    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:17.396391 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.396366    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"]
Jul 26 18:51:17.398493 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:51:17.398469    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70c2facf_9ed2_4729_9679_48a3c9add3c8.slice/crio-719e60304a9f7326c43e6256695f3f10ea666f469d5ede987a591ee614f5cd7a WatchSource:0}: Error finding container 719e60304a9f7326c43e6256695f3f10ea666f469d5ede987a591ee614f5cd7a: Status 404 returned error can't find the container with id 719e60304a9f7326c43e6256695f3f10ea666f469d5ede987a591ee614f5cd7a
Jul 26 18:51:17.998400 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.998366    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r" event={"ID":"70c2facf-9ed2-4729-9679-48a3c9add3c8","Type":"ContainerStarted","Data":"c722ec70261739319176f513376ba223bb03ccb7728dad99639fcd3bf5bfc734"}
Jul 26 18:51:17.998795 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:17.998424    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r" event={"ID":"70c2facf-9ed2-4729-9679-48a3c9add3c8","Type":"ContainerStarted","Data":"719e60304a9f7326c43e6256695f3f10ea666f469d5ede987a591ee614f5cd7a"}
Jul 26 18:51:18.013991 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:18.013946    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r" podStartSLOduration=2.013931918 podStartE2EDuration="2.013931918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:18.011760616 +0000 UTC m=+3232.201790370" watchObservedRunningTime="2026-07-26 18:51:18.013931918 +0000 UTC m=+3232.203961685"
Jul 26 18:51:19.004682 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:19.004651    2565 generic.go:356] "Generic (PLEG): container finished" podID="70c2facf-9ed2-4729-9679-48a3c9add3c8" containerID="c722ec70261739319176f513376ba223bb03ccb7728dad99639fcd3bf5bfc734" exitCode=0
Jul 26 18:51:19.005105 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:19.004733    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r" event={"ID":"70c2facf-9ed2-4729-9679-48a3c9add3c8","Type":"ContainerDied","Data":"c722ec70261739319176f513376ba223bb03ccb7728dad99639fcd3bf5bfc734"}
Jul 26 18:51:20.138184 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:20.138159    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:20.197096 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:20.197069    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c2facf-9ed2-4729-9679-48a3c9add3c8-maas-api-service-ca\") pod \"70c2facf-9ed2-4729-9679-48a3c9add3c8\" (UID: \"70c2facf-9ed2-4729-9679-48a3c9add3c8\") "
Jul 26 18:51:20.197424 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:20.197400    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70c2facf-9ed2-4729-9679-48a3c9add3c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "70c2facf-9ed2-4729-9679-48a3c9add3c8" (UID: "70c2facf-9ed2-4729-9679-48a3c9add3c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:20.298618 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:20.298529    2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/70c2facf-9ed2-4729-9679-48a3c9add3c8-maas-api-service-ca\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:21.016607 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:21.016580    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"
Jul 26 18:51:21.016812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:21.016631    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r" event={"ID":"70c2facf-9ed2-4729-9679-48a3c9add3c8","Type":"ContainerDied","Data":"719e60304a9f7326c43e6256695f3f10ea666f469d5ede987a591ee614f5cd7a"}
Jul 26 18:51:21.016812 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:21.016678    2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="719e60304a9f7326c43e6256695f3f10ea666f469d5ede987a591ee614f5cd7a"
Jul 26 18:51:26.738671 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:26.738635    2565 scope.go:117] "RemoveContainer" containerID="73264a45e57f5a35d325401273b5eca1d27cbae691780ce5078829aca50fb06c"
Jul 26 18:51:27.388687 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.388657    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"]
Jul 26 18:51:27.396488 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.396462    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-d112e4-29751525-r5d9l"]
Jul 26 18:51:27.420114 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.420075    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"]
Jul 26 18:51:27.420305 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.420280    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5" podUID="e25953ca-740d-4ca8-adb6-c2c08fcb7401" containerName="payload-processing" containerID="cri-o://bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e" gracePeriod=30
Jul 26 18:51:27.429073 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.429044    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"]
Jul 26 18:51:27.429288 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.429264    2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2" podUID="e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" containerName="payload-pre-processing" containerID="cri-o://562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59" gracePeriod=30
Jul 26 18:51:27.671067 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.671039    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:51:27.702455 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.702432    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:51:27.761069 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.761035    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkzw9\" (UniqueName: \"kubernetes.io/projected/e25953ca-740d-4ca8-adb6-c2c08fcb7401-kube-api-access-gkzw9\") pod \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") "
Jul 26 18:51:27.761529 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.761148    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25953ca-740d-4ca8-adb6-c2c08fcb7401-plugins-config-volume\") pod \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\" (UID: \"e25953ca-740d-4ca8-adb6-c2c08fcb7401\") "
Jul 26 18:51:27.761595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.761572    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e25953ca-740d-4ca8-adb6-c2c08fcb7401-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e25953ca-740d-4ca8-adb6-c2c08fcb7401" (UID: "e25953ca-740d-4ca8-adb6-c2c08fcb7401"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:27.763028 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.763008    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e25953ca-740d-4ca8-adb6-c2c08fcb7401-kube-api-access-gkzw9" (OuterVolumeSpecName: "kube-api-access-gkzw9") pod "e25953ca-740d-4ca8-adb6-c2c08fcb7401" (UID: "e25953ca-740d-4ca8-adb6-c2c08fcb7401"). InnerVolumeSpecName "kube-api-access-gkzw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:27.861895 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.861865    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-plugins-config-volume\") pod \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") "
Jul 26 18:51:27.862088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.861940    2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfjrh\" (UniqueName: \"kubernetes.io/projected/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-kube-api-access-jfjrh\") pod \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\" (UID: \"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f\") "
Jul 26 18:51:27.862088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.862057    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e25953ca-740d-4ca8-adb6-c2c08fcb7401-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:27.862088 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.862067    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkzw9\" (UniqueName: \"kubernetes.io/projected/e25953ca-740d-4ca8-adb6-c2c08fcb7401-kube-api-access-gkzw9\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:27.862273 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.862213    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" (UID: "e80772fc-f4fd-41b8-bb0d-6ba1012ae42f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:27.864076 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.864045    2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-kube-api-access-jfjrh" (OuterVolumeSpecName: "kube-api-access-jfjrh") pod "e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" (UID: "e80772fc-f4fd-41b8-bb0d-6ba1012ae42f"). InnerVolumeSpecName "kube-api-access-jfjrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:27.962574 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.962479    2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfjrh\" (UniqueName: \"kubernetes.io/projected/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-kube-api-access-jfjrh\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:27.962574 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:27.962514    2565 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f-plugins-config-volume\") on node \"ip-10-0-133-34.ec2.internal\" DevicePath \"\""
Jul 26 18:51:28.053926 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.053888    2565 generic.go:356] "Generic (PLEG): container finished" podID="e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" containerID="562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59" exitCode=0
Jul 26 18:51:28.054094 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.053951    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"
Jul 26 18:51:28.054094 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.053978    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2" event={"ID":"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f","Type":"ContainerDied","Data":"562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59"}
Jul 26 18:51:28.054094 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.054075    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2" event={"ID":"e80772fc-f4fd-41b8-bb0d-6ba1012ae42f","Type":"ContainerDied","Data":"d918def744cb5fda1fb8f8043381e60cf21304dcc152539af5aff58340e89767"}
Jul 26 18:51:28.054279 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.054109    2565 scope.go:117] "RemoveContainer" containerID="562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59"
Jul 26 18:51:28.055430 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.055257    2565 generic.go:356] "Generic (PLEG): container finished" podID="e25953ca-740d-4ca8-adb6-c2c08fcb7401" containerID="bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e" exitCode=0
Jul 26 18:51:28.055430 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.055314    2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"
Jul 26 18:51:28.055430 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.055312    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5" event={"ID":"e25953ca-740d-4ca8-adb6-c2c08fcb7401","Type":"ContainerDied","Data":"bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e"}
Jul 26 18:51:28.055430 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.055376    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5" event={"ID":"e25953ca-740d-4ca8-adb6-c2c08fcb7401","Type":"ContainerDied","Data":"d4c85f5e42628b18981d447f923119af86a1e563b72901734ac8b5c318651785"}
Jul 26 18:51:28.064034 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.064017    2565 scope.go:117] "RemoveContainer" containerID="562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59"
Jul 26 18:51:28.064303 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:51:28.064285    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59\": container with ID starting with 562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59 not found: ID does not exist" containerID="562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59"
Jul 26 18:51:28.064382 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.064311    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59"} err="failed to get container status \"562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59\": rpc error: code = NotFound desc = could not find container \"562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59\": container with ID starting with 562504fd9221a8b538bc481c1e8ab932a7c47b2d0b6fa91a0f4ee4ea549f0c59 not found: ID does not exist"
Jul 26 18:51:28.064382 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.064326    2565 scope.go:117] "RemoveContainer" containerID="bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e"
Jul 26 18:51:28.072264 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.072246    2565 scope.go:117] "RemoveContainer" containerID="bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e"
Jul 26 18:51:28.072513 ip-10-0-133-34 kubenswrapper[2565]: E0726 18:51:28.072494    2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e\": container with ID starting with bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e not found: ID does not exist" containerID="bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e"
Jul 26 18:51:28.072578 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.072519    2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e"} err="failed to get container status \"bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e\": rpc error: code = NotFound desc = could not find container \"bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e\": container with ID starting with bcd576e271d2a5c302340724f2d1936a59370d1921a9b1158a494627218c2a9e not found: ID does not exist"
Jul 26 18:51:28.077009 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.076988    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"]
Jul 26 18:51:28.081150 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.081127    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-d112e4-c4fdcf55b-npxx2"]
Jul 26 18:51:28.091859 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.091835    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"]
Jul 26 18:51:28.097982 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.097963    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-d112e4-6bcf44bb5d-lvsd5"]
Jul 26 18:51:28.318595 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.318567    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="617d082e-6d70-4f50-a2d1-c7ac5e410440" path="/var/lib/kubelet/pods/617d082e-6d70-4f50-a2d1-c7ac5e410440/volumes"
Jul 26 18:51:28.318909 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.318896    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e25953ca-740d-4ca8-adb6-c2c08fcb7401" path="/var/lib/kubelet/pods/e25953ca-740d-4ca8-adb6-c2c08fcb7401/volumes"
Jul 26 18:51:28.319206 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:28.319195    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" path="/var/lib/kubelet/pods/e80772fc-f4fd-41b8-bb0d-6ba1012ae42f/volumes"
Jul 26 18:51:30.536043 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:30.536003    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-bklf4"]
Jul 26 18:51:32.018499 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:32.018466    2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"]
Jul 26 18:51:32.020773 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:32.020753    2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-d112e4-lng2r"]
Jul 26 18:51:32.318902 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:32.318824    2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70c2facf-9ed2-4729-9679-48a3c9add3c8" path="/var/lib/kubelet/pods/70c2facf-9ed2-4729-9679-48a3c9add3c8/volumes"
Jul 26 18:51:44.443413 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:44.443367    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cxhmm_5ab06719-c410-4027-94ea-30365da8cfb7/manager/0.log"
Jul 26 18:51:45.113551 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:45.113526    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69cc94bb6c-xdsbf_71a8f8ca-4ce9-43d1-810b-21921235b7da/copy-manifests/0.log"
Jul 26 18:51:45.120017 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:45.119992    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69cc94bb6c-xdsbf_71a8f8ca-4ce9-43d1-810b-21921235b7da/manager/0.log"
Jul 26 18:51:45.680425 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:45.680398    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751510-g2dj6_dc0197cc-dee3-4a9d-b4bb-b28b9c0e5bca/cleanup/0.log"
Jul 26 18:51:45.785974 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:45.785947    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29751525-7cfnr_7d86370f-c438-439f-84b5-bd742717833f/cleanup/0.log"
Jul 26 18:51:46.632269 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.632225    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp_a86ef9c3-62ac-4c24-bab1-1864a788a037/pull/0.log"
Jul 26 18:51:46.638307 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.638283    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp_a86ef9c3-62ac-4c24-bab1-1864a788a037/extract/0.log"
Jul 26 18:51:46.643511 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.643482    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp_a86ef9c3-62ac-4c24-bab1-1864a788a037/util/0.log"
Jul 26 18:51:46.749804 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.749768    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt_fe602531-96a0-46f1-8d9f-0904ff48cc37/extract/0.log"
Jul 26 18:51:46.756031 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.756009    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt_fe602531-96a0-46f1-8d9f-0904ff48cc37/util/0.log"
Jul 26 18:51:46.761818 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.761792    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt_fe602531-96a0-46f1-8d9f-0904ff48cc37/pull/0.log"
Jul 26 18:51:46.864875 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.864847    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j_c78d29ad-052c-42ef-994d-c71bb0936496/util/0.log"
Jul 26 18:51:46.870489 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.870468    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j_c78d29ad-052c-42ef-994d-c71bb0936496/pull/0.log"
Jul 26 18:51:46.876799 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.876782    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j_c78d29ad-052c-42ef-994d-c71bb0936496/extract/0.log"
Jul 26 18:51:46.979532 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.979502    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4_6f73f2d9-9b81-4afa-ac6c-634a620a2568/util/0.log"
Jul 26 18:51:46.986130 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.986083    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4_6f73f2d9-9b81-4afa-ac6c-634a620a2568/pull/0.log"
Jul 26 18:51:46.991518 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:46.991501    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4_6f73f2d9-9b81-4afa-ac6c-634a620a2568/extract/0.log"
Jul 26 18:51:47.216646 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:47.216614    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wcqhn_864fac41-3d29-402f-ad83-0e16c41b4f2a/manager/0.log"
Jul 26 18:51:47.770151 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:47.770120    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bklf4_53db2922-3aa6-42af-abfc-f7400012ce5e/limitador/0.log"
Jul 26 18:51:47.884816 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:47.884789    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qdnfh_c10be1d9-754e-4151-8abc-c362270944c3/manager/0.log"
Jul 26 18:51:48.335563 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:48.335535    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6pkfm_c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c/discovery/0.log"
Jul 26 18:51:48.438347 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:48.438315    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-679f49c568-4rwtq_ed75bbf9-d69e-43ce-b29b-81acaa4e5213/kube-auth-proxy/0.log"
Jul 26 18:51:49.750828 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:49.750793    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-7jsnd_520ed8b5-bd68-4675-be8a-34f419ce2968/main/0.log"
Jul 26 18:51:57.794488 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:57.794445    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4k55t_3b6e1a55-1afc-40a7-be4c-9f5810802717/global-pull-secret-syncer/0.log"
Jul 26 18:51:57.959033 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:57.959001    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fkfch_a50ffe58-a86d-46f0-9feb-5d96623d19db/konnectivity-agent/0.log"
Jul 26 18:51:58.005638 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:51:58.005609    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-34.ec2.internal_bd7040ec0042cf6f2efb0378a077f8ac/haproxy/0.log"
Jul 26 18:52:02.534463 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.534433    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp_a86ef9c3-62ac-4c24-bab1-1864a788a037/extract/0.log"
Jul 26 18:52:02.559053 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.559021    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp_a86ef9c3-62ac-4c24-bab1-1864a788a037/util/0.log"
Jul 26 18:52:02.587054 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.587029    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598dsvp_a86ef9c3-62ac-4c24-bab1-1864a788a037/pull/0.log"
Jul 26 18:52:02.620653 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.620624    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt_fe602531-96a0-46f1-8d9f-0904ff48cc37/extract/0.log"
Jul 26 18:52:02.644843 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.644818    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt_fe602531-96a0-46f1-8d9f-0904ff48cc37/util/0.log"
Jul 26 18:52:02.672582 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.672557    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0f9wwt_fe602531-96a0-46f1-8d9f-0904ff48cc37/pull/0.log"
Jul 26 18:52:02.705795 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.705770    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j_c78d29ad-052c-42ef-994d-c71bb0936496/extract/0.log"
Jul 26 18:52:02.728211 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.728183    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j_c78d29ad-052c-42ef-994d-c71bb0936496/util/0.log"
Jul 26 18:52:02.756229 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.756204    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734w78j_c78d29ad-052c-42ef-994d-c71bb0936496/pull/0.log"
Jul 26 18:52:02.784279 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.784251    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4_6f73f2d9-9b81-4afa-ac6c-634a620a2568/extract/0.log"
Jul 26 18:52:02.811018 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.810946    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4_6f73f2d9-9b81-4afa-ac6c-634a620a2568/util/0.log"
Jul 26 18:52:02.836569 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:02.836542    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14hxj4_6f73f2d9-9b81-4afa-ac6c-634a620a2568/pull/0.log"
Jul 26 18:52:03.396460 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:03.396429    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wcqhn_864fac41-3d29-402f-ad83-0e16c41b4f2a/manager/0.log"
Jul 26 18:52:03.701496 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:03.701413    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-bklf4_53db2922-3aa6-42af-abfc-f7400012ce5e/limitador/0.log"
Jul 26 18:52:03.778661 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:03.778635    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qdnfh_c10be1d9-754e-4151-8abc-c362270944c3/manager/0.log"
Jul 26 18:52:05.901585 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:05.901549    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c4cjk_1620f426-1294-4f46-88f5-87cce4d3d73f/node-exporter/0.log"
Jul 26 18:52:05.924997 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:05.924969    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c4cjk_1620f426-1294-4f46-88f5-87cce4d3d73f/kube-rbac-proxy/0.log"
Jul 26 18:52:05.947985 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:05.947960    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c4cjk_1620f426-1294-4f46-88f5-87cce4d3d73f/init-textfile/0.log"
Jul 26 18:52:08.240247 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240214    2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"]
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240495    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e25953ca-740d-4ca8-adb6-c2c08fcb7401" containerName="payload-processing"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240506    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e25953ca-740d-4ca8-adb6-c2c08fcb7401" containerName="payload-processing"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240521    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" containerName="payload-pre-processing"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240527    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" containerName="payload-pre-processing"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240534    2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70c2facf-9ed2-4729-9679-48a3c9add3c8" containerName="revoke-keys"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240540    2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="70c2facf-9ed2-4729-9679-48a3c9add3c8" containerName="revoke-keys"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240588    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e25953ca-740d-4ca8-adb6-c2c08fcb7401" containerName="payload-processing"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240597    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="70c2facf-9ed2-4729-9679-48a3c9add3c8" containerName="revoke-keys"
Jul 26 18:52:08.240630 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.240605    2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e80772fc-f4fd-41b8-bb0d-6ba1012ae42f" containerName="payload-pre-processing"
Jul 26 18:52:08.243732 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.243691    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.246829 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.246808    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lz7b2\"/\"kube-root-ca.crt\""
Jul 26 18:52:08.247850 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.247828    2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lz7b2\"/\"default-dockercfg-chqtw\""
Jul 26 18:52:08.247943 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.247833    2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lz7b2\"/\"openshift-service-ca.crt\""
Jul 26 18:52:08.251690 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.251671    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"]
Jul 26 18:52:08.295597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.295560    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-proc\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.295597 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.295595    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-sys\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.295846 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.295621    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8z74\" (UniqueName: \"kubernetes.io/projected/ea600fb5-ee8c-4469-b306-ce5765cbceaa-kube-api-access-g8z74\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.295846 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.295676    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-lib-modules\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.295846 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.295734    2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-podres\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.396985 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.396946    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-sys\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.396985 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.396985    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8z74\" (UniqueName: \"kubernetes.io/projected/ea600fb5-ee8c-4469-b306-ce5765cbceaa-kube-api-access-g8z74\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397009    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-lib-modules\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397035    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-podres\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397084    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-sys\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397152    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-lib-modules\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397160    2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-proc\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397198 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397193    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-podres\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.397397 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.397192    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ea600fb5-ee8c-4469-b306-ce5765cbceaa-proc\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.405084 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.405057    2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8z74\" (UniqueName: \"kubernetes.io/projected/ea600fb5-ee8c-4469-b306-ce5765cbceaa-kube-api-access-g8z74\") pod \"perf-node-gather-daemonset-blscf\" (UID: \"ea600fb5-ee8c-4469-b306-ce5765cbceaa\") " pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.553930 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.553843    2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:08.684861 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:08.684838    2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"]
Jul 26 18:52:08.686890 ip-10-0-133-34 kubenswrapper[2565]: W0726 18:52:08.686864    2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podea600fb5_ee8c_4469_b306_ce5765cbceaa.slice/crio-89ad3c131764b73e69329292877d713f551f610b6dfd6768f463b4f7bacbd6b1 WatchSource:0}: Error finding container 89ad3c131764b73e69329292877d713f551f610b6dfd6768f463b4f7bacbd6b1: Status 404 returned error can't find the container with id 89ad3c131764b73e69329292877d713f551f610b6dfd6768f463b4f7bacbd6b1
Jul 26 18:52:09.268530 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:09.268492    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf" event={"ID":"ea600fb5-ee8c-4469-b306-ce5765cbceaa","Type":"ContainerStarted","Data":"eec9bc48b92949115b0509665c489d5d027bb47bd0e90ef86200c1a1162971a8"}
Jul 26 18:52:09.268984 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:09.268545    2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf" event={"ID":"ea600fb5-ee8c-4469-b306-ce5765cbceaa","Type":"ContainerStarted","Data":"89ad3c131764b73e69329292877d713f551f610b6dfd6768f463b4f7bacbd6b1"}
Jul 26 18:52:09.268984 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:09.268645    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:09.287576 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:09.287521    2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf" podStartSLOduration=1.287505522 podStartE2EDuration="1.287505522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:52:09.28653924 +0000 UTC m=+3283.476568993" watchObservedRunningTime="2026-07-26 18:52:09.287505522 +0000 UTC m=+3283.477535275"
Jul 26 18:52:10.922267 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:10.922238    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rlbpg_9e52b638-70d2-4be3-99f0-0f5b36322b7f/dns/0.log"
Jul 26 18:52:10.946999 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:10.946971    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rlbpg_9e52b638-70d2-4be3-99f0-0f5b36322b7f/kube-rbac-proxy/0.log"
Jul 26 18:52:10.975273 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:10.975242    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7jpg7_17b413f2-3a11-4e6e-bd9e-2c72c576749d/dns-node-resolver/0.log"
Jul 26 18:52:11.629793 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:11.629765    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pqvrw_85696f9e-3458-467e-bf4b-9503978d99ca/node-ca/0.log"
Jul 26 18:52:12.728410 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:12.728378    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6pkfm_c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c/discovery/0.log"
Jul 26 18:52:12.752782 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:12.752757    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-679f49c568-4rwtq_ed75bbf9-d69e-43ce-b29b-81acaa4e5213/kube-auth-proxy/0.log"
Jul 26 18:52:13.739291 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:13.739246    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6jtsk_23eee869-252e-495d-9a41-efaf13e2e3e6/serve-healthcheck-canary/0.log"
Jul 26 18:52:14.369332 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:14.369302    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kr997_6eff04b4-327d-4f73-b801-698564387ab4/kube-rbac-proxy/0.log"
Jul 26 18:52:14.391628 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:14.391599    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kr997_6eff04b4-327d-4f73-b801-698564387ab4/exporter/0.log"
Jul 26 18:52:14.415983 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:14.415960    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kr997_6eff04b4-327d-4f73-b801-698564387ab4/extractor/0.log"
Jul 26 18:52:15.283138 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:15.283111    2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lz7b2/perf-node-gather-daemonset-blscf"
Jul 26 18:52:16.877860 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:16.877821    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cxhmm_5ab06719-c410-4027-94ea-30365da8cfb7/manager/0.log"
Jul 26 18:52:17.335115 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:17.335089    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69cc94bb6c-xdsbf_71a8f8ca-4ce9-43d1-810b-21921235b7da/manager/0.log"
Jul 26 18:52:17.358284 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:17.358262    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-69cc94bb6c-xdsbf_71a8f8ca-4ce9-43d1-810b-21921235b7da/copy-manifests/0.log"
Jul 26 18:52:19.026295 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:19.026264    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6pkfm_c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c/discovery/0.log"
Jul 26 18:52:19.038037 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:19.038014    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-679f49c568-4rwtq_ed75bbf9-d69e-43ce-b29b-81acaa4e5213/kube-auth-proxy/0.log"
Jul 26 18:52:20.753172 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:20.753137    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-6pkfm_c0ac5cdb-425c-49ad-8e40-a4dba9bcc07c/discovery/0.log"
Jul 26 18:52:20.764596 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:20.764557    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-679f49c568-4rwtq_ed75bbf9-d69e-43ce-b29b-81acaa4e5213/kube-auth-proxy/0.log"
Jul 26 18:52:23.899507 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:23.899478    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tp9dq_568409b6-709f-4564-b94a-4aa912e208e6/migrator/0.log"
Jul 26 18:52:23.921575 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:23.921554    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-tp9dq_568409b6-709f-4564-b94a-4aa912e208e6/graceful-termination/0.log"
Jul 26 18:52:25.602094 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.602066    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/kube-multus-additional-cni-plugins/0.log"
Jul 26 18:52:25.623677 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.623647    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/egress-router-binary-copy/0.log"
Jul 26 18:52:25.645201 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.645172    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/cni-plugins/0.log"
Jul 26 18:52:25.667081 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.667056    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/bond-cni-plugin/0.log"
Jul 26 18:52:25.689285 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.689260    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/routeoverride-cni/0.log"
Jul 26 18:52:25.711650 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.711624    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/whereabouts-cni-bincopy/0.log"
Jul 26 18:52:25.734673 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.734647    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ljtcc_06937257-8927-494e-a4a6-532585fdc8e8/whereabouts-cni/0.log"
Jul 26 18:52:25.850355 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.850323    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rtr8b_e3562f0e-f4c2-447d-a176-0596c7b205bd/kube-multus/0.log"
Jul 26 18:52:25.926803 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.926715    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nthlf_caba9223-8c40-4e7e-9740-98fcb97755b0/network-metrics-daemon/0.log"
Jul 26 18:52:25.945095 ip-10-0-133-34 kubenswrapper[2565]: I0726 18:52:25.945069    2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nthlf_caba9223-8c40-4e7e-9740-98fcb97755b0/kube-rbac-proxy/0.log"