Jul 28 01:02:11.208849 ip-10-0-131-236 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 01:02:11.208883 ip-10-0-131-236 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 01:02:11.208893 ip-10-0-131-236 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 01:02:11.209271 ip-10-0-131-236 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 01:02:21.341794 ip-10-0-131-236 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 01:02:21.341813 ip-10-0-131-236 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot df18d84cc4704913919910a85a5609de --
Jul 28 01:04:23.866438 ip-10-0-131-236 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 01:04:24.286346 ip-10-0-131-236 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 01:04:24.286346 ip-10-0-131-236 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 01:04:24.286346 ip-10-0-131-236 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 01:04:24.286346 ip-10-0-131-236 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 01:04:24.286346 ip-10-0-131-236 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 01:04:24.287622 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.287102    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 01:04:24.292501 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292476    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:24.292501 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292500    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:24.292501 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292504    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292507    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292510    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292513    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292516    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292520    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292523    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292526    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292528    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292531    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292533    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292536    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292538    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292541    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292544    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292547    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292549    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292552    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292555    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:24.292601 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292562    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292566    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292571    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292573    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292577    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292579    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292582    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292585    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292587    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292590    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292592    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292595    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292598    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292600    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292603    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292606    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292609    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292612    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292615    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292617    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:24.293068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292620    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292623    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292626    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292628    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292632    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292637    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292641    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292643    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292646    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292649    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292651    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292654    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292656    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292659    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292661    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292664    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292667    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292670    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292672    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:24.293542 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292675    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292678    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292680    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292684    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292686    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292689    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292692    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292695    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292699    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292701    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292704    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292707    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292709    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292712    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292714    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292718    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292720    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292723    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292725    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292729    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:24.294074 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292731    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292734    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292736    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292739    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292741    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292744    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292746    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292749    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292752    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292754    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292757    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292759    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292763    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292765    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292768    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292771    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292773    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292777    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.292780    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293551    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:24.294571 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293558    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293562    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293565    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293568    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293570    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293574    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293577    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293579    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293582    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293585    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293588    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293591    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293594    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293597    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293600    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293602    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293605    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293607    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293610    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293613    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:24.295082 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293615    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293618    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293623    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293626    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293628    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293633    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293636    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293640    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293643    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293645    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293648    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293650    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293653    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293655    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293658    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293661    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293663    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293666    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293668    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:24.295597 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293671    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293674    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293676    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293679    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293681    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293683    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293687    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293691    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293693    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293696    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293698    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293700    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293704    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293706    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293709    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293712    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293714    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293717    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293719    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293722    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:24.296087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293724    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293727    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293729    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293732    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293736    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293738    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293740    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293743    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293745    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293747    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293750    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293753    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293755    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293757    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293760    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293762    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293765    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293767    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293770    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293773    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:24.296578 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293775    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293778    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293780    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293782    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293785    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293788    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293790    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293793    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293796    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293799    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293801    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293804    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293807    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293809    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293812    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293815    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293819    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293821    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.293824    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:24.297087 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294345    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294352    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294356    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294359    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294361    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294364    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294367    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294371    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294373    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294376    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294378    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294381    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294385    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294387    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294390    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294392    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294396    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294400    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294403    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:24.297545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294406    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294409    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294412    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294415    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294418    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294421    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294424    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294427    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294430    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294433    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294436    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294439    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294441    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294444    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294447    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294449    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294452    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294454    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294457    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294460    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:24.298024 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294462    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294465    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294468    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294470    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294472    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294475    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294484    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294486    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294488    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294491    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294493    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294496    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294498    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294500    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294503    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294506    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294508    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294511    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294513    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:24.298549 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294515    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294520    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294523    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294526    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294529    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294532    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294534    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294537    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294540    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294543    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294545    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294551    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294554    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294556    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294559    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294561    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294564    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294566    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294569    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:24.299034 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294571    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294575    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294577    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294581    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294584    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294586    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294589    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294592    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294595    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294598    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294600    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294603    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294605    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294607    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294610    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294612    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294615    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294617    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294621    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294625    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:24.299495 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294628    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.294630    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294705    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294713    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294721    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294726    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294730    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294733    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294747    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294751    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294754    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294757    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294761    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294764    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294768    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294772    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294775    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294778    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294781    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294784    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294791    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294794    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294797    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294800    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 01:04:24.300014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294803    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294808    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294810    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294813    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294817    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294820    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294823    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294826    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294829    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294832    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294836    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294839    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294843    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294846    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294849    2566 flags.go:64] FLAG: --enable-server="true"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294852    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294856    2566 flags.go:64] FLAG: --event-burst="100"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294860    2566 flags.go:64] FLAG: --event-qps="50"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294863    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294866    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294869    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294873    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294877    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294880    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294883    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 28 01:04:24.300603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294886    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294889    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294892    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294896    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294899    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294902    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294905    2566 flags.go:64] FLAG: --feature-gates=""
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294909    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294912    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294916    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294919    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294922    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294925    2566 flags.go:64] FLAG: --help="false"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294928    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294931    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294934    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294937    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294940    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294943    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294948    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294950    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294953    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294956    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 01:04:24.301232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294959    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294962    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294978    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294981    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294984    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294988    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294990    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294993    2566 flags.go:64] FLAG: --lock-file=""
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.294997    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295000    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295003    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295009    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295013    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295016    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295018    2566 flags.go:64] FLAG: --logging-format="text"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295021    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295025    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295028    2566 flags.go:64] FLAG: --manifest-url=""
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295031    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295038    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295041    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295046    2566 flags.go:64] FLAG: --max-pods="110"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295049    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295052    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295055    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 01:04:24.301832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295058    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295061    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295064    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295068    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295076    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295079    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295082    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295085    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295089    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295094    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295097    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295101    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295104    2566 flags.go:64] FLAG: --port="10250"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295106    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295109    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-019a4d612e7faf005"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295113    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295116    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295119    2566 flags.go:64] FLAG: --register-node="true"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295122    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295130    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295133    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295136    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295139    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295143    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295146    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 01:04:24.302497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295149    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295152    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295155    2566 flags.go:64] FLAG: --runonce="false"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295158    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295161    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295178    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295182    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295186    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295189    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295192    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295195    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295199    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295202    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295205    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295208    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295211    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295214    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295217    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295222    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295225    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295228    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295233    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295236    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295240    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295245    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 01:04:24.303122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295248    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295253    2566 flags.go:64] FLAG: --v="2"
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295258    2566 flags.go:64] FLAG: --version="false"
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295262    2566 flags.go:64] FLAG: --vmodule=""
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295266    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295269    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295398    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295403    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295406    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295409    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295411    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295414    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295416    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295419    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295422    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295427    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295429    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295432    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295436    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295439    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:24.303731 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295441    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295444    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295446    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295449    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295451    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295454    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295456    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295459    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295463    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295467    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295470    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295473    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295476    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295481    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295484    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295487    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295490    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295492    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295495    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:24.304254 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295498    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295500    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295503    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295505    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295508    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295511    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295513    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295516    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295526    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295531    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295534    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295538    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295540    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295543    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295545    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295548    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295550    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295553    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295555    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295558    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:24.304726 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295560    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295563    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295566    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295568    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295571    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295573    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295578    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295581    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295583    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295586    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295588    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295591    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295593    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295596    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295599    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295602    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295605    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295609    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295612    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295615    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:24.305322 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295618    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295620    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295623    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295628    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295631    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295633    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295636    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295639    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295642    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295644    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295647    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295649    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295652    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295655    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295657    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295660    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295662    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295665    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295669    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295672    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:24.305816 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295675    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295677    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295680    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295682    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295685    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.295687    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.295693    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.303089    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.303114    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.303137    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.303145    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303211    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303218    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303221    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:24.306331 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303225    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303228    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303231    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303233    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303236    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303238    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303242    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303246    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303249    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303252    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303255    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303258    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303260    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303263    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303266    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303269    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303271    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303273    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303276    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303278    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:24.306734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303281    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303284    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303288    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303291    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303294    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303296    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303306    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303309    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303311    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303314    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303317    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303319    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303322    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303324    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303327    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303329    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303332    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303334    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303336    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:24.307249 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303339    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303342    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303344    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303347    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303349    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303352    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303354    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303357    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303359    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303362    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303364    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303367    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303369    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303372    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303375    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303377    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303380    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303382    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303385    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303388    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:24.307734 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303397    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303399    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303402    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303405    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303407    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303410    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303412    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303415    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303418    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303420    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303422    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303425    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303428    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303431    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303433    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303435    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303438    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303440    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303443    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:24.308391 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303447    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303451    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303453    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303456    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303459    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303462    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303465    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303467    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303470    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303473    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303476    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303479    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303482    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303484    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303492    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303495    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303498    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:24.309210 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303500    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.303506    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303688    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303695    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303699    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303702    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303704    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303707    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303710    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303714    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303718    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303721    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303723    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303726    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 01:04:24.309663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303729    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303731    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303734    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303737    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303740    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303742    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303745    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303747    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303750    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303752    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303755    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303757    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303760    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303764    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303768    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303771    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303780    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303783    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303785    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303788    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 01:04:24.310071 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303790    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303793    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303795    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303798    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303800    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303803    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303805    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303808    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303810    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303812    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303815    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303817    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303820    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303822    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303824    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303827    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303829    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303832    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303834    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303836    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 01:04:24.310849 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303839    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303841    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303844    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303846    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303849    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303852    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303855    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303857    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303859    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303868    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303870    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303873    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303875    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303878    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303880    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303883    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303885    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303887    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303890    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303892    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 01:04:24.311514 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303895    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303903    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303906    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303908    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303911    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303913    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303916    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303918    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303921    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303923    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303926    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303928    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303931    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303933    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303936    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303938    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303941    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303943    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303946    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303949    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 01:04:24.312220 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303951    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303954    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303981    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303985    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303988    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303990    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:24.303994    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.303999    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.304152    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.306253    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.307935    2566 server.go:1019] "Starting client certificate rotation"
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.308080    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 01:04:24.312725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.308143    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 01:04:24.313032 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.312684    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 01:04:24.316371 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.316352    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 01:04:24.327208 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.327182    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 28 01:04:24.329461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.329441    2566 log.go:25] "Validated CRI v1 image API"
Jul 28 01:04:24.330271 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.330256    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 01:04:24.332723 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.332701    2566 fs.go:135] Filesystem UUIDs: map[4601e59c-c204-4d4c-be5c-4610ee2ba84c:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 dc55d6ae-ebbd-4b3c-83b8-5266297f9cc2:/dev/nvme0n1p3]
Jul 28 01:04:24.332795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.332720    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 01:04:24.338085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.338061    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 01:04:24.338562 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.338449    2566 manager.go:216] Machine: {Timestamp:2026-07-28 01:04:24.337272241 +0000 UTC m=+0.348510841 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107307 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0c6730e00c7f5fd191568a637e68 SystemUUID:ec2d0c67-30e0-0c7f-5fd1-91568a637e68 BootID:df18d84c-c470-4913-9199-10a85a5609de Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0c:8c:dd:86:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0c:8c:dd:86:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:1c:06:1b:32:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 01:04:24.338562 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.338562    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 01:04:24.338663 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.338656    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 01:04:24.341740 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.341714    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 01:04:24.341897 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.341745    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-236.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 01:04:24.344191 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.344173    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 01:04:24.344191 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.344193    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 01:04:24.344271 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.344202    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 01:04:24.344859 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.344843    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 01:04:24.345628 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.345615    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 01:04:24.348030 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348011    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 01:04:24.348196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348186    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 01:04:24.348238 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348231    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 01:04:24.348266 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348245    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 01:04:24.348266 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348262    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 01:04:24.348358 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348277    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 28 01:04:24.348358 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348290    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 01:04:24.348814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348798    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 01:04:24.348897 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.348819    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 01:04:24.351099 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.351077    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 01:04:24.354119 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.354091    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 01:04:24.354285 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.354258    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 01:04:24.355926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355907    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 01:04:24.355926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355927    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355933    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355938    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355947    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355953    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355961    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355985    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355993    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.355999    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.356008    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 01:04:24.356049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.356013    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 01:04:24.357008 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.356998    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 01:04:24.357008 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.357007    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 01:04:24.359716 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.359692    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-236.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 01:04:24.359814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.359795    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j7wfp"
Jul 28 01:04:24.360271 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.360251    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-236.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 01:04:24.360324 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.360250    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 01:04:24.360461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.360450    2566 server.go:1268] "Started kubelet"
Jul 28 01:04:24.360617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.360575    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 01:04:24.360679 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.360642    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 01:04:24.360756 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.360598    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 01:04:24.361465 ip-10-0-131-236 systemd[1]: Started Kubernetes Kubelet.
Jul 28 01:04:24.362212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.362198    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 01:04:24.364944 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.364930    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 01:04:24.367380 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.367341    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 01:04:24.367596 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.367578    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j7wfp"
Jul 28 01:04:24.368555 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.368535    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 01:04:24.368555 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.368543    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 01:04:24.368716 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.368647    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.369062 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369044    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 01:04:24.369161 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369145    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 01:04:24.369221 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369164    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 01:04:24.369325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369312    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 01:04:24.369325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369324    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 01:04:24.369449 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369438    2566 factory.go:55] Registering systemd factory
Jul 28 01:04:24.369499 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369455    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 28 01:04:24.369857 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369833    2566 factory.go:153] Registering CRI-O factory
Jul 28 01:04:24.369857 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369849    2566 factory.go:223] Registration of the crio container factory successfully
Jul 28 01:04:24.370011 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369921    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 01:04:24.370011 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369945    2566 factory.go:103] Registering Raw factory
Jul 28 01:04:24.370011 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.369989    2566 manager.go:1196] Started watching for new ooms in manager
Jul 28 01:04:24.370332 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.369414    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-236.ec2.internal.18c64e8bbbfe7c9e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-236.ec2.internal,UID:ip-10-0-131-236.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-236.ec2.internal,},FirstTimestamp:2026-07-28 01:04:24.360418462 +0000 UTC m=+0.371657061,LastTimestamp:2026-07-28 01:04:24.360418462 +0000 UTC m=+0.371657061,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-236.ec2.internal,}"
Jul 28 01:04:24.370567 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.370554    2566 manager.go:349] Starting recovery of all containers
Jul 28 01:04:24.376108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.376083    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:24.378737 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.378707    2566 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-236.ec2.internal\" not found" node="ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.379021 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.378947    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 01:04:24.382599 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.382393    2566 manager.go:354] Recovery completed
Jul 28 01:04:24.384914 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.384889    2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 28 01:04:24.388262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.388240    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.390257 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390239    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.390329 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390273    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.390329 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390284    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.390829 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390812    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 01:04:24.390829 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390828    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 01:04:24.390899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390846    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 01:04:24.390899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.390863    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.391667 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.391650    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.391728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.391680    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.391728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.391695    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.392545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.392533    2566 policy_none.go:49] "None policy: Start"
Jul 28 01:04:24.392590 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.392550    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 01:04:24.392590 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.392561    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 01:04:24.393626 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.393616    2566 policy_none.go:47] "Start"
Jul 28 01:04:24.442724 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.442700    2566 manager.go:337] "Starting Device Plugin manager"
Jul 28 01:04:24.442883 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.442753    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 01:04:24.442883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.442764    2566 server.go:87] "Starting device plugin registration server"
Jul 28 01:04:24.443073 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.443059    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 01:04:24.443125 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.443076    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 01:04:24.443173 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.443160    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 01:04:24.443515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.443248    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 01:04:24.443515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.443258    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 01:04:24.443816 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.443793    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 01:04:24.443876 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.443851    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.543568 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.543476    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.544688 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.544609    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.544688 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.544649    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.544688 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.544665    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.544870 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.544702    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.550014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.549992    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.550119 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.550021    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-236.ec2.internal\": node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.554244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.554222    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 01:04:24.554370 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.554266    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 01:04:24.554370 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.554286    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 01:04:24.554370 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.554320    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 01:04:24.556091 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.556072    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:24.563627 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.563597    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.654682 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.654643    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"]
Jul 28 01:04:24.654846 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.654744    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.656433 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.656414    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.656519 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.656452    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.656519 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.656463    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.657669 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.657654    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.657808 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.657780    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.657899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.657828    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.658498 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.658478    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.658604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.658502    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.658604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.658510    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.658604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.658520    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.658604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.658524    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.658604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.658535    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.659800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.659787    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.659854 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.659814    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 01:04:24.660487 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.660473    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 01:04:24.660546 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.660502    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 01:04:24.660546 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.660514    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeHasSufficientPID"
Jul 28 01:04:24.663695 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.663675    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.671094 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.671068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09244df59df110fdf5d72f4c51556192-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"09244df59df110fdf5d72f4c51556192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.671209 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.671101    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09244df59df110fdf5d72f4c51556192-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"09244df59df110fdf5d72f4c51556192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.671209 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.671129    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3d4f2c0da36356c1d7f9784538048ba-config\") pod \"kube-apiserver-proxy-ip-10-0-131-236.ec2.internal\" (UID: \"b3d4f2c0da36356c1d7f9784538048ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.683952 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.683927    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-236.ec2.internal\" not found" node="ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.688884 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.688865    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-236.ec2.internal\" not found" node="ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.763910 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.763876    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.771232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.771201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09244df59df110fdf5d72f4c51556192-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"09244df59df110fdf5d72f4c51556192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.771332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.771239    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3d4f2c0da36356c1d7f9784538048ba-config\") pod \"kube-apiserver-proxy-ip-10-0-131-236.ec2.internal\" (UID: \"b3d4f2c0da36356c1d7f9784538048ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.771332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.771258    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09244df59df110fdf5d72f4c51556192-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"09244df59df110fdf5d72f4c51556192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.771332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.771284    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/09244df59df110fdf5d72f4c51556192-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"09244df59df110fdf5d72f4c51556192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.771332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.771295    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09244df59df110fdf5d72f4c51556192-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal\" (UID: \"09244df59df110fdf5d72f4c51556192\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.771332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.771308    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b3d4f2c0da36356c1d7f9784538048ba-config\") pod \"kube-apiserver-proxy-ip-10-0-131-236.ec2.internal\" (UID: \"b3d4f2c0da36356c1d7f9784538048ba\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.864773 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.864729    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.965312 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:24.965270    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:24.986819 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.986773    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:24.992324 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:24.992293    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:25.065784 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:25.065732    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:25.167318 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:25.167233    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:25.267593 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:25.267552    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:25.307263 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.307228    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 01:04:25.307835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.307397    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 01:04:25.307835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.307439    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 01:04:25.309035 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.309018    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:25.367776 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:25.367739    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-236.ec2.internal\" not found"
Jul 28 01:04:25.372969 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.372932    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 00:59:25 +0000 UTC" deadline="2028-01-31 11:53:32.602428866 +0000 UTC"
Jul 28 01:04:25.373105 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.372967    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13258h49m7.229467051s"
Jul 28 01:04:25.399741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.399714    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:25.468912 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.468827    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:25.481672 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.481641    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 01:04:25.482660 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.482640    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal"
Jul 28 01:04:25.492593 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.492561    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 01:04:25.579308 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:25.579273    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d4f2c0da36356c1d7f9784538048ba.slice/crio-7044cef7feb899a109a015b0a34b08d17e82d083ac01991e95a6f101518c5069 WatchSource:0}: Error finding container 7044cef7feb899a109a015b0a34b08d17e82d083ac01991e95a6f101518c5069: Status 404 returned error can't find the container with id 7044cef7feb899a109a015b0a34b08d17e82d083ac01991e95a6f101518c5069
Jul 28 01:04:25.579890 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:25.579863    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09244df59df110fdf5d72f4c51556192.slice/crio-e23778a2ffc62ab319d09f68ef20cf8eea7c421ed685e3a71178e3fbf9aacf47 WatchSource:0}: Error finding container e23778a2ffc62ab319d09f68ef20cf8eea7c421ed685e3a71178e3fbf9aacf47: Status 404 returned error can't find the container with id e23778a2ffc62ab319d09f68ef20cf8eea7c421ed685e3a71178e3fbf9aacf47
Jul 28 01:04:25.585132 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:25.585108    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:04:26.199222 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.199171    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:26.299478 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.299435    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 01:04:26.357384 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.357345    2566 apiserver.go:52] "Watching apiserver"
Jul 28 01:04:26.367322 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.367277    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 01:04:26.369495 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.369466    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 01:04:26.370815 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.370783    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-rh6sg","openshift-network-operator/iptables-alerter-z9sck","openshift-ovn-kubernetes/ovnkube-node-j49pf","kube-system/konnectivity-agent-v28qs","kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal","openshift-image-registry/node-ca-n6jdz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal","openshift-multus/network-metrics-daemon-fnd2j","openshift-network-diagnostics/network-check-target-v4sm2","kube-system/global-pull-secret-syncer-hdkfq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj","openshift-cluster-node-tuning-operator/tuned-sh4hj","openshift-dns/node-resolver-58nwp","openshift-multus/multus-6ln99"]
Jul 28 01:04:26.372708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.372680    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.372837 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.372763    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:26.374159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.374127    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.375553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.375528    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.376935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.376908    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 01:04:26.377314 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.377292    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.377400 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.377329    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zhhkc\""
Jul 28 01:04:26.377551 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.377533    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.378099 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.377751    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.378424 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.378402    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 01:04:26.379612 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.378771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.379923 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.379897    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.380743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.380723    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.380939 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.380921    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381197    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381399    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381509    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381585    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x4597\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381696    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381775    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381803    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.381892    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f9d7t\""
Jul 28 01:04:26.382078 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.381887    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:26.383286 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383004    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.383438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383300    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.383438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383342    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lnhhg\""
Jul 28 01:04:26.383438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383414    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 01:04:26.383438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383430    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vwvpj\""
Jul 28 01:04:26.384211 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383753    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 01:04:26.384211 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.383931    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.384211 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.384033    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 01:04:26.384211 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.384094    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 01:04:26.384453 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.384224    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.384453 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.384309    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.385088 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.385069    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.385967 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.385949    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.386548 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.386529    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.386711 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.386694    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.386895 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.386852    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-v9ckx\""
Jul 28 01:04:26.387884 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.387856    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:26.387981 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.387941    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:26.388708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.388431    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8grww\""
Jul 28 01:04:26.388708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.388432    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.388708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.388479    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 01:04:26.388708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.388556    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.388930 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.388744    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 01:04:26.389515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.389497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8fmqh\""
Jul 28 01:04:26.389758 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.389738    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 01:04:26.390308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.390285    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.392678 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.392658    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:04:26.392777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.392755    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9qwtx\""
Jul 28 01:04:26.392913 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.392897    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 01:04:26.408515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.408481    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5pttb"
Jul 28 01:04:26.417984 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.417955    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5pttb"
Jul 28 01:04:26.470879 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.470851    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 01:04:26.481026 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.480960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-systemd-units\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481026 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481024    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29001342-cb93-42de-a039-2e4558104d8f-multus-daemon-config\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.481247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cni-binary-copy\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.481247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-registration-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.481247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-os-release\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.481247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-system-cni-dir\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.481247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481175    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.481247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481203    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcwgx\" (UniqueName: \"kubernetes.io/projected/7bef27d0-03da-4866-90aa-3dbe5df2e392-kube-api-access-xcwgx\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrdsm\" (UniqueName: \"kubernetes.io/projected/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-kube-api-access-wrdsm\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481288    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-ovnkube-script-lib\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-device-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-cni-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysctl-conf\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8443e39b-0f3d-423c-ad49-0ab06a0140e8-host\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481385    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481401    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-run-netns\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-node-log\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481431    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-env-overrides\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-cni-multus\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.481468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481464    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-conf-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481479    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db9c66a8-6859-4e59-a9ec-938b424484f0-hosts-file\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-host\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-etc-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-ovn\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481577    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481602    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de3d9941-c334-46bc-8d9e-65cfc52464ce-host-slash\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481627    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-system-cni-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cnibin\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-cni-netd\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481716    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de3d9941-c334-46bc-8d9e-65cfc52464ce-iptables-alerter-script\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481732    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nr5n\" (UniqueName: \"kubernetes.io/projected/de3d9941-c334-46bc-8d9e-65cfc52464ce-kube-api-access-6nr5n\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481748    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm44s\" (UniqueName: \"kubernetes.io/projected/0fb4f15a-ca09-41c6-86dc-d99584578a2e-kube-api-access-zm44s\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481771    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-sys\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-lib-modules\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.481828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481825    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-var-lib-kubelet\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-cni-bin\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7l9f\" (UniqueName: \"kubernetes.io/projected/130ab43f-f510-4070-a8f7-5479505cf32d-kube-api-access-q7l9f\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481940    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-os-release\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.481982    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1388316-7ecb-428d-8e92-a983372bdf5b-konnectivity-ca\") pod \"konnectivity-agent-v28qs\" (UID: \"e1388316-7ecb-428d-8e92-a983372bdf5b\") " pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482009    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-cni-bin\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-multus-certs\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482131    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-etc-selinux\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482266    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-cnibin\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482313    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-k8s-cni-cncf-io\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysconfig\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.482404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8443e39b-0f3d-423c-ad49-0ab06a0140e8-serviceca\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-slash\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482515    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-ovnkube-config\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-kubelet\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482582    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h2g5\" (UniqueName: \"kubernetes.io/projected/db9c66a8-6859-4e59-a9ec-938b424484f0-kube-api-access-4h2g5\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482609    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-modprobe-d\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482661    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-tuned\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482703    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bef27d0-03da-4866-90aa-3dbe5df2e392-tmp\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-sys-fs\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482766    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29001342-cb93-42de-a039-2e4558104d8f-cni-binary-copy\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482790    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-systemd\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482828    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-socket-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482852    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-systemd\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.482909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482882    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-hostroot\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-etc-kubernetes\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.482974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483046    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c86d027-c224-4ab8-8b80-36d85be6e41b-kubelet-config\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-kubelet\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c86d027-c224-4ab8-8b80-36d85be6e41b-dbus\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-netns\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483150    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrh6k\" (UniqueName: \"kubernetes.io/projected/29001342-cb93-42de-a039-2e4558104d8f-kube-api-access-qrh6k\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483174    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9c66a8-6859-4e59-a9ec-938b424484f0-tmp-dir\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483199    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1388316-7ecb-428d-8e92-a983372bdf5b-agent-certs\") pod \"konnectivity-agent-v28qs\" (UID: \"e1388316-7ecb-428d-8e92-a983372bdf5b\") " pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-var-lib-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483268    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483300    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-log-socket\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-socket-dir-parent\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483349    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-run\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.483600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483373    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/130ab43f-f510-4070-a8f7-5479505cf32d-ovn-node-metrics-cert\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.484172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-kubernetes\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.484172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8lnk\" (UniqueName: \"kubernetes.io/projected/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-kube-api-access-m8lnk\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.484172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqfq5\" (UniqueName: \"kubernetes.io/projected/8443e39b-0f3d-423c-ad49-0ab06a0140e8-kube-api-access-bqfq5\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.484172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-run-ovn-kubernetes\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.484172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.483501    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysctl-d\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.560092 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.560037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal" event={"ID":"b3d4f2c0da36356c1d7f9784538048ba","Type":"ContainerStarted","Data":"7044cef7feb899a109a015b0a34b08d17e82d083ac01991e95a6f101518c5069"}
Jul 28 01:04:26.561285 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.561252    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" event={"ID":"09244df59df110fdf5d72f4c51556192","Type":"ContainerStarted","Data":"e23778a2ffc62ab319d09f68ef20cf8eea7c421ed685e3a71178e3fbf9aacf47"}
Jul 28 01:04:26.583861 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.583806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-kubernetes\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.583876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8lnk\" (UniqueName: \"kubernetes.io/projected/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-kube-api-access-m8lnk\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.583906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqfq5\" (UniqueName: \"kubernetes.io/projected/8443e39b-0f3d-423c-ad49-0ab06a0140e8-kube-api-access-bqfq5\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.583931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-run-ovn-kubernetes\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.583935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-kubernetes\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.583954    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysctl-d\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-systemd-units\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29001342-cb93-42de-a039-2e4558104d8f-multus-daemon-config\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584057    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cni-binary-copy\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.584085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584078    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-registration-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-os-release\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584114    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysctl-d\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584129    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-system-cni-dir\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584185    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcwgx\" (UniqueName: \"kubernetes.io/projected/7bef27d0-03da-4866-90aa-3dbe5df2e392-kube-api-access-xcwgx\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrdsm\" (UniqueName: \"kubernetes.io/projected/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-kube-api-access-wrdsm\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584239    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-ovnkube-script-lib\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584262    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-device-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584285    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-cni-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584307    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysctl-conf\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584340    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8443e39b-0f3d-423c-ad49-0ab06a0140e8-host\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584363    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-run-netns\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-node-log\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.584426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584431    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-env-overrides\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584458    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-cni-multus\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-conf-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584508    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db9c66a8-6859-4e59-a9ec-938b424484f0-hosts-file\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-host\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-etc-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-ovn\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584607    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de3d9941-c334-46bc-8d9e-65cfc52464ce-host-slash\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584686    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-system-cni-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-run-ovn-kubernetes\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584728    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-system-cni-dir\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584789    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-systemd-units\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584842    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysctl-conf\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cnibin\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-cni-netd\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de3d9941-c334-46bc-8d9e-65cfc52464ce-iptables-alerter-script\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.585214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584959    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nr5n\" (UniqueName: \"kubernetes.io/projected/de3d9941-c334-46bc-8d9e-65cfc52464ce-kube-api-access-6nr5n\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.584987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm44s\" (UniqueName: \"kubernetes.io/projected/0fb4f15a-ca09-41c6-86dc-d99584578a2e-kube-api-access-zm44s\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585015    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-sys\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-lib-modules\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585096    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-var-lib-kubelet\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585127    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-cni-bin\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7l9f\" (UniqueName: \"kubernetes.io/projected/130ab43f-f510-4070-a8f7-5479505cf32d-kube-api-access-q7l9f\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-etc-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-os-release\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585230    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-registration-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585240    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1388316-7ecb-428d-8e92-a983372bdf5b-konnectivity-ca\") pod \"konnectivity-agent-v28qs\" (UID: \"e1388316-7ecb-428d-8e92-a983372bdf5b\") " pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-cni-bin\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8443e39b-0f3d-423c-ad49-0ab06a0140e8-host\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585302    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/db9c66a8-6859-4e59-a9ec-938b424484f0-hosts-file\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-multus-certs\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585341    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-multus-certs\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-host\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-etc-selinux\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-cnibin\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585403    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-run-netns\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-k8s-cni-cncf-io\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585423    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585440    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29001342-cb93-42de-a039-2e4558104d8f-multus-daemon-config\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585449    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-etc-selinux\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-node-log\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-ovn\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.585502    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-conf-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585512    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysconfig\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585447    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-sysconfig\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585539    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585558    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8443e39b-0f3d-423c-ad49-0ab06a0140e8-serviceca\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585582    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de3d9941-c334-46bc-8d9e-65cfc52464ce-host-slash\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.586980 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.585615    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:27.085566611 +0000 UTC m=+3.096805217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585638    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cni-binary-copy\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585646    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-system-cni-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cnibin\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585720    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-slash\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585741    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-cni-netd\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-ovnkube-config\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585770    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-device-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585782    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-kubelet\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h2g5\" (UniqueName: \"kubernetes.io/projected/db9c66a8-6859-4e59-a9ec-938b424484f0-kube-api-access-4h2g5\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-cni-dir\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-modprobe-d\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-tuned\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bef27d0-03da-4866-90aa-3dbe5df2e392-tmp\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-sys-fs\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585983    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8443e39b-0f3d-423c-ad49-0ab06a0140e8-serviceca\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585986    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29001342-cb93-42de-a039-2e4558104d8f-cni-binary-copy\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.587896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586026    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-ovnkube-script-lib\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-systemd\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586084    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-socket-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-systemd\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586150    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-systemd\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586201    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-systemd\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-hostroot\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586245    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586304    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-cni-multus\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-sys\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586336    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-socket-dir\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-hostroot\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.586382    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586399    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0fb4f15a-ca09-41c6-86dc-d99584578a2e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586408    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/de3d9941-c334-46bc-8d9e-65cfc52464ce-iptables-alerter-script\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-etc-kubernetes\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.589244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586433    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-etc-kubernetes\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.586445    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:04:27.086426985 +0000 UTC m=+3.097665575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.585208    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-os-release\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-kubelet\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586472    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-var-lib-cni-bin\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586474    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-lib-modules\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586506    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29001342-cb93-42de-a039-2e4558104d8f-cni-binary-copy\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586517    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586537    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-var-lib-kubelet\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-cni-bin\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586578    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-os-release\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-slash\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586659    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-sys-fs\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c86d027-c224-4ab8-8b80-36d85be6e41b-kubelet-config\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586693    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-kubelet\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-modprobe-d\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.590108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586709    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-cnibin\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c86d027-c224-4ab8-8b80-36d85be6e41b-dbus\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586742    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0fb4f15a-ca09-41c6-86dc-d99584578a2e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586750    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-netns\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-k8s-cni-cncf-io\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586757    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-ovnkube-config\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586794    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-host-run-netns\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586795    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-host-kubelet\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrh6k\" (UniqueName: \"kubernetes.io/projected/29001342-cb93-42de-a039-2e4558104d8f-kube-api-access-qrh6k\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586884    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2c86d027-c224-4ab8-8b80-36d85be6e41b-dbus\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586910    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586943    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2c86d027-c224-4ab8-8b80-36d85be6e41b-kubelet-config\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.586947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9c66a8-6859-4e59-a9ec-938b424484f0-tmp-dir\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1388316-7ecb-428d-8e92-a983372bdf5b-agent-certs\") pod \"konnectivity-agent-v28qs\" (UID: \"e1388316-7ecb-428d-8e92-a983372bdf5b\") " pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-var-lib-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587076    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587101    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-log-socket\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-var-lib-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.590910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587137    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-socket-dir-parent\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587166    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-run\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587193    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/130ab43f-f510-4070-a8f7-5479505cf32d-ovn-node-metrics-cert\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587222    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-log-socket\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587245    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e1388316-7ecb-428d-8e92-a983372bdf5b-konnectivity-ca\") pod \"konnectivity-agent-v28qs\" (UID: \"e1388316-7ecb-428d-8e92-a983372bdf5b\") " pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587279    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/130ab43f-f510-4070-a8f7-5479505cf32d-run-openvswitch\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587319    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29001342-cb93-42de-a039-2e4558104d8f-multus-socket-dir-parent\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587334    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9c66a8-6859-4e59-a9ec-938b424484f0-tmp-dir\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587335    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7bef27d0-03da-4866-90aa-3dbe5df2e392-run\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.587456    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/130ab43f-f510-4070-a8f7-5479505cf32d-env-overrides\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.590817    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7bef27d0-03da-4866-90aa-3dbe5df2e392-tmp\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.590881    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7bef27d0-03da-4866-90aa-3dbe5df2e392-etc-tuned\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.590993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/130ab43f-f510-4070-a8f7-5479505cf32d-ovn-node-metrics-cert\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.591553 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.591274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e1388316-7ecb-428d-8e92-a983372bdf5b-agent-certs\") pod \"konnectivity-agent-v28qs\" (UID: \"e1388316-7ecb-428d-8e92-a983372bdf5b\") " pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.594782 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.594677    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:26.594782 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.594704    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:26.594782 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.594719    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:26.594782 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:26.594781    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:27.094760852 +0000 UTC m=+3.105999465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:26.600425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.600390    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h2g5\" (UniqueName: \"kubernetes.io/projected/db9c66a8-6859-4e59-a9ec-938b424484f0-kube-api-access-4h2g5\") pod \"node-resolver-58nwp\" (UID: \"db9c66a8-6859-4e59-a9ec-938b424484f0\") " pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.601718 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.601591    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm44s\" (UniqueName: \"kubernetes.io/projected/0fb4f15a-ca09-41c6-86dc-d99584578a2e-kube-api-access-zm44s\") pod \"multus-additional-cni-plugins-rh6sg\" (UID: \"0fb4f15a-ca09-41c6-86dc-d99584578a2e\") " pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.601718 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.601691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nr5n\" (UniqueName: \"kubernetes.io/projected/de3d9941-c334-46bc-8d9e-65cfc52464ce-kube-api-access-6nr5n\") pod \"iptables-alerter-z9sck\" (UID: \"de3d9941-c334-46bc-8d9e-65cfc52464ce\") " pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.601892 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.601757    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrh6k\" (UniqueName: \"kubernetes.io/projected/29001342-cb93-42de-a039-2e4558104d8f-kube-api-access-qrh6k\") pod \"multus-6ln99\" (UID: \"29001342-cb93-42de-a039-2e4558104d8f\") " pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.601892 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.601802    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8lnk\" (UniqueName: \"kubernetes.io/projected/6cc8f4ff-056a-4ce8-b7f2-6a6103387762-kube-api-access-m8lnk\") pod \"aws-ebs-csi-driver-node-2s2jj\" (UID: \"6cc8f4ff-056a-4ce8-b7f2-6a6103387762\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.602343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.602322    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrdsm\" (UniqueName: \"kubernetes.io/projected/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-kube-api-access-wrdsm\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:26.602579 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.602558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcwgx\" (UniqueName: \"kubernetes.io/projected/7bef27d0-03da-4866-90aa-3dbe5df2e392-kube-api-access-xcwgx\") pod \"tuned-sh4hj\" (UID: \"7bef27d0-03da-4866-90aa-3dbe5df2e392\") " pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:26.602986 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.602897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7l9f\" (UniqueName: \"kubernetes.io/projected/130ab43f-f510-4070-a8f7-5479505cf32d-kube-api-access-q7l9f\") pod \"ovnkube-node-j49pf\" (UID: \"130ab43f-f510-4070-a8f7-5479505cf32d\") " pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.604090 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.604033    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqfq5\" (UniqueName: \"kubernetes.io/projected/8443e39b-0f3d-423c-ad49-0ab06a0140e8-kube-api-access-bqfq5\") pod \"node-ca-n6jdz\" (UID: \"8443e39b-0f3d-423c-ad49-0ab06a0140e8\") " pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.692679 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.692640    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj"
Jul 28 01:04:26.701218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.701187    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:26.711374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.711327    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:26.717468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.717439    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6ln99"
Jul 28 01:04:26.726499 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.726451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-n6jdz"
Jul 28 01:04:26.734526 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.734491    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-58nwp"
Jul 28 01:04:26.743534 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.743497    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-z9sck"
Jul 28 01:04:26.752476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.752440    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rh6sg"
Jul 28 01:04:26.766639 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:26.766599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sh4hj"
Jul 28 01:04:27.091957 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:27.091912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:27.092171 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:27.091980    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:27.092171 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.092097    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:27.092171 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.092106    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:27.092303 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.092177    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:04:28.092154207 +0000 UTC m=+4.103392815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:27.092303 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.092198    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:28.092187539 +0000 UTC m=+4.103426127 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:27.192953 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:27.192894    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:27.193186 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.193121    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:27.193186 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.193148    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:27.193186 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.193163    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:27.193345 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:27.193237    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:28.193217866 +0000 UTC m=+4.204456473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:27.421521 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:27.421424    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 00:59:27 +0000 UTC" deadline="2028-02-26 08:20:03.691489396 +0000 UTC"
Jul 28 01:04:27.421521 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:27.421463    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13879h15m36.27003117s"
Jul 28 01:04:27.639585 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.639429    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8443e39b_0f3d_423c_ad49_0ab06a0140e8.slice/crio-54c4d8b6fd5fd4a914afa736870bc7108fc8f371eef2fbe3ef272e61bb162227 WatchSource:0}: Error finding container 54c4d8b6fd5fd4a914afa736870bc7108fc8f371eef2fbe3ef272e61bb162227: Status 404 returned error can't find the container with id 54c4d8b6fd5fd4a914afa736870bc7108fc8f371eef2fbe3ef272e61bb162227
Jul 28 01:04:27.641523 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.641387    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9c66a8_6859_4e59_a9ec_938b424484f0.slice/crio-50db4b5aa4856393dc94a24630075589b9ed81971a3a88f9e3fcaf6bd91be4cc WatchSource:0}: Error finding container 50db4b5aa4856393dc94a24630075589b9ed81971a3a88f9e3fcaf6bd91be4cc: Status 404 returned error can't find the container with id 50db4b5aa4856393dc94a24630075589b9ed81971a3a88f9e3fcaf6bd91be4cc
Jul 28 01:04:27.642590 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.642517    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3d9941_c334_46bc_8d9e_65cfc52464ce.slice/crio-26d76771dfb777fed39d91488ac10f959c88b903abeaaa30363d498e76e9f207 WatchSource:0}: Error finding container 26d76771dfb777fed39d91488ac10f959c88b903abeaaa30363d498e76e9f207: Status 404 returned error can't find the container with id 26d76771dfb777fed39d91488ac10f959c88b903abeaaa30363d498e76e9f207
Jul 28 01:04:27.649263 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.649229    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc8f4ff_056a_4ce8_b7f2_6a6103387762.slice/crio-3e6d0c8c1c1c595f1a47dce0e013799b3cbdbff481b7d15975ced21627b8959b WatchSource:0}: Error finding container 3e6d0c8c1c1c595f1a47dce0e013799b3cbdbff481b7d15975ced21627b8959b: Status 404 returned error can't find the container with id 3e6d0c8c1c1c595f1a47dce0e013799b3cbdbff481b7d15975ced21627b8959b
Jul 28 01:04:27.657930 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.657894    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1388316_7ecb_428d_8e92_a983372bdf5b.slice/crio-69897952c3057134567c3f55903a6776e59cf69bfc833e4e64021e1fdff7335d WatchSource:0}: Error finding container 69897952c3057134567c3f55903a6776e59cf69bfc833e4e64021e1fdff7335d: Status 404 returned error can't find the container with id 69897952c3057134567c3f55903a6776e59cf69bfc833e4e64021e1fdff7335d
Jul 28 01:04:27.658778 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.658749    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130ab43f_f510_4070_a8f7_5479505cf32d.slice/crio-c46738988b46bd29773c8896c0dda5a0e370754412d23e8e5991abd62b048d3c WatchSource:0}: Error finding container c46738988b46bd29773c8896c0dda5a0e370754412d23e8e5991abd62b048d3c: Status 404 returned error can't find the container with id c46738988b46bd29773c8896c0dda5a0e370754412d23e8e5991abd62b048d3c
Jul 28 01:04:27.660354 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.660329    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bef27d0_03da_4866_90aa_3dbe5df2e392.slice/crio-570f538126c06dfaaf320fdbcba8f08b18fdffe30e2fef13ac191b06da903226 WatchSource:0}: Error finding container 570f538126c06dfaaf320fdbcba8f08b18fdffe30e2fef13ac191b06da903226: Status 404 returned error can't find the container with id 570f538126c06dfaaf320fdbcba8f08b18fdffe30e2fef13ac191b06da903226
Jul 28 01:04:27.661333 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:04:27.661294    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29001342_cb93_42de_a039_2e4558104d8f.slice/crio-a342ef9abf66b51fe8a73030ae0fe752a4255391912355928e17190353c757c8 WatchSource:0}: Error finding container a342ef9abf66b51fe8a73030ae0fe752a4255391912355928e17190353c757c8: Status 404 returned error can't find the container with id a342ef9abf66b51fe8a73030ae0fe752a4255391912355928e17190353c757c8
Jul 28 01:04:28.101360 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.101068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:28.101532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.101396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:28.101609 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.101572    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:28.101660 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.101638    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:04:30.101618101 +0000 UTC m=+6.112856702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:28.102166 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.102145    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:28.102253 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.102201    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:30.102184394 +0000 UTC m=+6.113422991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:28.202164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.202119    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:28.202363 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.202322    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:28.202363 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.202341    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:28.202363 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.202354    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:28.202527 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.202439    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:30.202417559 +0000 UTC m=+6.213656171 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:28.422402 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.422238    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 00:59:27 +0000 UTC" deadline="2028-05-03 12:01:41.22037665 +0000 UTC"
Jul 28 01:04:28.422402 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.422309    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15490h57m12.79807276s"
Jul 28 01:04:28.556213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.556177    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:28.556392 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.556325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:28.556455 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.556436    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:28.556514 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.556324    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:28.556514 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.556194    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:28.556643 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:28.556574    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:28.581824 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.581781    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"c46738988b46bd29773c8896c0dda5a0e370754412d23e8e5991abd62b048d3c"}
Jul 28 01:04:28.588795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.588702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sh4hj" event={"ID":"7bef27d0-03da-4866-90aa-3dbe5df2e392","Type":"ContainerStarted","Data":"570f538126c06dfaaf320fdbcba8f08b18fdffe30e2fef13ac191b06da903226"}
Jul 28 01:04:28.607385 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.607340    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj" event={"ID":"6cc8f4ff-056a-4ce8-b7f2-6a6103387762","Type":"ContainerStarted","Data":"3e6d0c8c1c1c595f1a47dce0e013799b3cbdbff481b7d15975ced21627b8959b"}
Jul 28 01:04:28.615523 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.615478    2566 generic.go:356] "Generic (PLEG): container finished" podID="09244df59df110fdf5d72f4c51556192" containerID="7ccb704984d10832e07875b39c109754d8ea0e4b70873ab77aff162c7fbe27fa" exitCode=0
Jul 28 01:04:28.615702 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.615580    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" event={"ID":"09244df59df110fdf5d72f4c51556192","Type":"ContainerDied","Data":"7ccb704984d10832e07875b39c109754d8ea0e4b70873ab77aff162c7fbe27fa"}
Jul 28 01:04:28.628953 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.628916    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v28qs" event={"ID":"e1388316-7ecb-428d-8e92-a983372bdf5b","Type":"ContainerStarted","Data":"69897952c3057134567c3f55903a6776e59cf69bfc833e4e64021e1fdff7335d"}
Jul 28 01:04:28.644822 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.644702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9sck" event={"ID":"de3d9941-c334-46bc-8d9e-65cfc52464ce","Type":"ContainerStarted","Data":"26d76771dfb777fed39d91488ac10f959c88b903abeaaa30363d498e76e9f207"}
Jul 28 01:04:28.676457 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.676417    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-58nwp" event={"ID":"db9c66a8-6859-4e59-a9ec-938b424484f0","Type":"ContainerStarted","Data":"50db4b5aa4856393dc94a24630075589b9ed81971a3a88f9e3fcaf6bd91be4cc"}
Jul 28 01:04:28.686660 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.686617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n6jdz" event={"ID":"8443e39b-0f3d-423c-ad49-0ab06a0140e8","Type":"ContainerStarted","Data":"54c4d8b6fd5fd4a914afa736870bc7108fc8f371eef2fbe3ef272e61bb162227"}
Jul 28 01:04:28.709354 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.706248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal" event={"ID":"b3d4f2c0da36356c1d7f9784538048ba","Type":"ContainerStarted","Data":"df94e66e75b24e005a4572c8c562df638508ab22788ddd8251680a995f4b6208"}
Jul 28 01:04:28.716171 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.716079    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerStarted","Data":"f334701a7030ae2eaad8b3a3f5057b90345e4305df31aa3de4e37758f6afd9f2"}
Jul 28 01:04:28.729450 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:28.729405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6ln99" event={"ID":"29001342-cb93-42de-a039-2e4558104d8f","Type":"ContainerStarted","Data":"a342ef9abf66b51fe8a73030ae0fe752a4255391912355928e17190353c757c8"}
Jul 28 01:04:29.751619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:29.751519    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" event={"ID":"09244df59df110fdf5d72f4c51556192","Type":"ContainerStarted","Data":"a74c43d8204e04e7152470b88a7a684f938979dd3d5dfb953574da92b5d83e27"}
Jul 28 01:04:29.769274 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:29.768700    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-236.ec2.internal" podStartSLOduration=3.768678097 podStartE2EDuration="3.768678097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:28.725430107 +0000 UTC m=+4.736668720" watchObservedRunningTime="2026-07-28 01:04:29.768678097 +0000 UTC m=+5.779916711"
Jul 28 01:04:30.120498 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:30.120452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:30.120706 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:30.120526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:30.120706 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.120669    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:30.120811 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.120736    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:04:34.120716546 +0000 UTC m=+10.131955141 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:30.121196 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.121176    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:30.121312 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.121234    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:34.121216423 +0000 UTC m=+10.132455018 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:30.222102 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:30.222037    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:30.222308 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.222291    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:30.222372 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.222318    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:30.222372 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.222332    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:30.222475 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.222395    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:34.222374259 +0000 UTC m=+10.233612864 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:30.554815 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:30.554725    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:30.554992 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.554894    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:30.555440 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:30.555386    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:30.555557 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.555526    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:30.555637 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:30.555595    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:30.558668 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:30.555689    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:32.560104 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:32.559543    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:32.560104 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:32.559692    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:32.561101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:32.560771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:32.561101 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:32.560879    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:32.561101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:32.560942    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:32.561101 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:32.561036    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:34.158993 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:34.158938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:34.159510 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:34.159063    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:34.159510 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.159212    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:34.159510 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.159281    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:04:42.1592609 +0000 UTC m=+18.170499493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:34.159510 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.159357    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:34.159510 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.159395    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:42.159382046 +0000 UTC m=+18.170620638 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:34.260251 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:34.260197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:34.260915 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.260408    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:34.260915 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.260444    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:34.260915 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.260461    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:34.260915 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.260535    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:42.260514068 +0000 UTC m=+18.271752662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:34.555610 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:34.555527    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:34.555756 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.555687    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:34.559696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:34.556112    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:34.559696 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.556209    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:34.559696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:34.556270    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:34.559696 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:34.556342    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:36.558850 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:36.558819    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:36.559341 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:36.558940    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:36.559341 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:36.559141    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:36.559341 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:36.559235    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:36.559341 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:36.559262    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:36.559341 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:36.559315    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:38.554770 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:38.554733    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:38.554770 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:38.554764    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:38.555306 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:38.554861    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:38.555306 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:38.554885    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:38.555306 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:38.554982    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:38.555306 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:38.555030    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:40.555624 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:40.555582    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:40.556116 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:40.555606    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:40.556116 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:40.555720    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:40.556116 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:40.555726    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:40.556116 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:40.555822    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:40.556116 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:40.555956    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:42.224231 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:42.224191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:42.224777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:42.224279    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:42.224777 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.224353    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:42.224777 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.224383    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:42.224777 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.224440    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:04:58.224419142 +0000 UTC m=+34.235657731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:42.224777 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.224458    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:58.224450634 +0000 UTC m=+34.235689221 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:42.325529 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:42.325492    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:42.325715 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.325648    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:42.325715 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.325672    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:42.325715 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.325683    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:42.325847 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.325743    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:04:58.32572449 +0000 UTC m=+34.336963091 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:42.554855 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:42.554772    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:42.555043 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:42.554900    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:42.555043 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.554921    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:42.555043 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.555014    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:42.555240 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:42.555061    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:42.555240 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:42.555139    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:44.556220 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:44.556178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:44.556631 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:44.556266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:44.556631 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:44.556285    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:44.556631 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:44.556381    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:44.556631 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:44.556440    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:44.556631 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:44.556513    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:46.554613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:46.554571    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:46.554613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:46.554593    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:46.555197 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:46.554745    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:46.555197 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:46.554823    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:46.555197 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:46.554851    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:46.555197 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:46.554959    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:47.790920 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.790417    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v28qs" event={"ID":"e1388316-7ecb-428d-8e92-a983372bdf5b","Type":"ContainerStarted","Data":"541d581fc15fd0291257b34ca76c86cddfb02ca3686e9f82f461181958314628"}
Jul 28 01:04:47.792546 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.792520    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-58nwp" event={"ID":"db9c66a8-6859-4e59-a9ec-938b424484f0","Type":"ContainerStarted","Data":"d5e3df551d336dfb0bf248c41b2e9b55fa46550d5d9d0e3a2b13d97d8494713a"}
Jul 28 01:04:47.794639 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.794597    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-n6jdz" event={"ID":"8443e39b-0f3d-423c-ad49-0ab06a0140e8","Type":"ContainerStarted","Data":"cd6c972aca9fa92b580d0ca0a21eb86231342c42450e3981dffbcc7f6410d79d"}
Jul 28 01:04:47.796247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.796221    2566 generic.go:356] "Generic (PLEG): container finished" podID="0fb4f15a-ca09-41c6-86dc-d99584578a2e" containerID="5b1030df3edf118d5aef0b75f51572c9890e6324e89d28350ab4b87802eb12ef" exitCode=0
Jul 28 01:04:47.796367 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.796254    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerDied","Data":"5b1030df3edf118d5aef0b75f51572c9890e6324e89d28350ab4b87802eb12ef"}
Jul 28 01:04:47.798305 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.798231    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6ln99" event={"ID":"29001342-cb93-42de-a039-2e4558104d8f","Type":"ContainerStarted","Data":"845bdc94a3cd8fa2a2bb8bf1ad5a7641f2d2d9a6da3f2574b9a12815ce50da17"}
Jul 28 01:04:47.801898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.801873    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"27216afe78aafef02edbaefa20c7580c3b9f8cb3d3bf2cd96e50695a144013da"}
Jul 28 01:04:47.802016 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.801950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"98f60607cec3215eaa59906134e042462bb7186f7c328cf1e37c7d0adf968ea7"}
Jul 28 01:04:47.802084 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.802025    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"24344b3e41b4b8a3831dc51f1cbd31dbaaae321fd4cc27777f7fcd6492ebfa25"}
Jul 28 01:04:47.802084 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.802075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"71add60bcf0ecb9d42f85aaf3b7f221851a25150a43d0a54164359fe9af409eb"}
Jul 28 01:04:47.802279 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.802126    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"fa07cd73545eff19bb13e5e10dc303ed9df2fdf5eb349509fa68afa3670443d6"}
Jul 28 01:04:47.802279 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.802169    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"7bfac1a583b4010d4a6c3e25ed5b1a9e9bd0e8968cfe69984c7af23be0dd4439"}
Jul 28 01:04:47.803439 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.803415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sh4hj" event={"ID":"7bef27d0-03da-4866-90aa-3dbe5df2e392","Type":"ContainerStarted","Data":"fc3c0550c6c595307f6b67c1b6c1ed6c725c319e3379f673275ed35ddb686936"}
Jul 28 01:04:47.805275 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.805230    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-236.ec2.internal" podStartSLOduration=21.80519464 podStartE2EDuration="21.80519464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:29.769202464 +0000 UTC m=+5.780441078" watchObservedRunningTime="2026-07-28 01:04:47.80519464 +0000 UTC m=+23.816433248"
Jul 28 01:04:47.805631 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.805605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj" event={"ID":"6cc8f4ff-056a-4ce8-b7f2-6a6103387762","Type":"ContainerStarted","Data":"d43968f220b02f17f8cbe5dbb8f4a562a33369babc53c0d2ff68135f92301ae7"}
Jul 28 01:04:47.806136 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.806097    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v28qs" podStartSLOduration=7.5957142 podStartE2EDuration="22.806081449s" totalImagesPullingTime="15.210367249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:47.805367049 +0000 UTC m=+23.816605659" watchObservedRunningTime="2026-07-28 01:04:47.806081449 +0000 UTC m=+23.817320059"
Jul 28 01:04:47.839938 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.839883    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sh4hj" podStartSLOduration=3.475389434 podStartE2EDuration="22.839864s" totalImagesPullingTime="19.364474566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:47.839556202 +0000 UTC m=+23.850794809" watchObservedRunningTime="2026-07-28 01:04:47.839864 +0000 UTC m=+23.851102608"
Jul 28 01:04:47.855595 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.855550    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6ln99" podStartSLOduration=3.436516605 podStartE2EDuration="22.855535715s" totalImagesPullingTime="19.41901911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:47.855366501 +0000 UTC m=+23.866605110" watchObservedRunningTime="2026-07-28 01:04:47.855535715 +0000 UTC m=+23.866774325"
Jul 28 01:04:47.869301 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.868962    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-n6jdz" podStartSLOduration=3.485130614 podStartE2EDuration="22.868942104s" totalImagesPullingTime="19.38381149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:47.868285349 +0000 UTC m=+23.879523960" watchObservedRunningTime="2026-07-28 01:04:47.868942104 +0000 UTC m=+23.880180714"
Jul 28 01:04:47.885284 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:47.882842    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-58nwp" podStartSLOduration=3.506525154 podStartE2EDuration="22.882818597s" totalImagesPullingTime="19.376293443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:47.882366791 +0000 UTC m=+23.893605400" watchObservedRunningTime="2026-07-28 01:04:47.882818597 +0000 UTC m=+23.894057207"
Jul 28 01:04:48.554916 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.554884    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:48.555144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.554884    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:48.555144 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:48.555008    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:48.555144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.555041    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:48.555144 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:48.555133    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:48.555345 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:48.555218    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:48.579317 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.579287    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 01:04:48.810231 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.810148    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj" event={"ID":"6cc8f4ff-056a-4ce8-b7f2-6a6103387762","Type":"ContainerStarted","Data":"a282a2a295fd0dac6d9f7f42d351676905edc65187d8445d1fa24bf3de12b933"}
Jul 28 01:04:48.811716 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.811684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-z9sck" event={"ID":"de3d9941-c334-46bc-8d9e-65cfc52464ce","Type":"ContainerStarted","Data":"40e935034d3a7ace71199a63085be13f0e62b881498bed7bc38e9275670e12b9"}
Jul 28 01:04:48.829261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:48.829202    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-z9sck" podStartSLOduration=4.453291254 podStartE2EDuration="23.829181522s" totalImagesPullingTime="19.375890268s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:48.828765029 +0000 UTC m=+24.840003639" watchObservedRunningTime="2026-07-28 01:04:48.829181522 +0000 UTC m=+24.840420131"
Jul 28 01:04:49.457904 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:49.457787    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T01:04:48.579307252Z","UUID":"cfc59446-d308-430c-864e-d11f153470a4","Handler":null,"Name":"","Endpoint":""}
Jul 28 01:04:49.459442 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:49.459413    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 01:04:49.459585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:49.459462    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 01:04:49.821642 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:49.821608    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"8a217f418649eef184bc032b55d62821bc070882c84b5a1841ddb70fea68e34e"}
Jul 28 01:04:49.824623 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:49.824555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj" event={"ID":"6cc8f4ff-056a-4ce8-b7f2-6a6103387762","Type":"ContainerStarted","Data":"d5ba95534f27449ff8cab94b7efba731c622e924c4fd4e8e86ab875c2552eabc"}
Jul 28 01:04:49.840617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:49.840567    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2s2jj" podStartSLOduration=3.194712149 podStartE2EDuration="24.840549609s" totalImagesPullingTime="21.64583746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:49.839841044 +0000 UTC m=+25.851079644" watchObservedRunningTime="2026-07-28 01:04:49.840549609 +0000 UTC m=+25.851788217"
Jul 28 01:04:50.555355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:50.555313    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:50.555355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:50.555344    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:50.555603 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:50.555321    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:50.555603 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:50.555502    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:50.555603 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:50.555587    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:50.555774 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:50.555686    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:50.852609 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:50.852577    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:50.853408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:50.853381    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:51.829950 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:51.829735    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:51.830337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:51.830317    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v28qs"
Jul 28 01:04:52.554698 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.554668    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:52.555314 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.554708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:52.555314 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.554668    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:52.555314 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:52.554791    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:52.555314 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:52.554874    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:52.555314 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:52.554981    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:52.834718 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.834627    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" event={"ID":"130ab43f-f510-4070-a8f7-5479505cf32d","Type":"ContainerStarted","Data":"3c6223f9548656c46d910fda202978fa0592da1e7f7db04a6df49a688dae4d2f"}
Jul 28 01:04:52.835008 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.834989    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:52.837250 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.837227    2566 generic.go:356] "Generic (PLEG): container finished" podID="0fb4f15a-ca09-41c6-86dc-d99584578a2e" containerID="b229e329a38c0a9106d3d13545a938cbaebfac80debc817ab2ae75b2d11ac911" exitCode=0
Jul 28 01:04:52.837361 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.837303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerDied","Data":"b229e329a38c0a9106d3d13545a938cbaebfac80debc817ab2ae75b2d11ac911"}
Jul 28 01:04:52.851204 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.851171    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:52.860264 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:52.860220    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf" podStartSLOduration=8.42885843 podStartE2EDuration="27.860206038s" totalImagesPullingTime="19.431347608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:04:52.859498027 +0000 UTC m=+28.870736638" watchObservedRunningTime="2026-07-28 01:04:52.860206038 +0000 UTC m=+28.871444653"
Jul 28 01:04:53.839334 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:53.839145    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 28 01:04:53.839800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:53.839306    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:53.854913 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:53.854889    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:54.385375 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.385275    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:04:54.465076 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.465045    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v4sm2"]
Jul 28 01:04:54.465253 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.465169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:54.465306 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:54.465258    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:54.468268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.468180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hdkfq"]
Jul 28 01:04:54.468419 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.468282    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:54.468419 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:54.468384    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:54.469103 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.469079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fnd2j"]
Jul 28 01:04:54.469212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.469178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:54.469315 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:54.469293    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:54.843709 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.843677    2566 generic.go:356] "Generic (PLEG): container finished" podID="0fb4f15a-ca09-41c6-86dc-d99584578a2e" containerID="cb2608b3f886bde5491478bd6b79ae14ef01e112fef86dfcd0589358ec016b1a" exitCode=0
Jul 28 01:04:54.844165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:54.843762    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerDied","Data":"cb2608b3f886bde5491478bd6b79ae14ef01e112fef86dfcd0589358ec016b1a"}
Jul 28 01:04:55.554963 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:55.554904    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:55.555185 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:55.554962    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:55.555185 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:55.555053    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:55.555185 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:55.555113    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:55.848957 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:55.848921    2566 generic.go:356] "Generic (PLEG): container finished" podID="0fb4f15a-ca09-41c6-86dc-d99584578a2e" containerID="6bc526085f62762ea4931709dbac708584f1c42a0c84f9a9a955bbdc1e9dd80f" exitCode=0
Jul 28 01:04:55.849355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:55.848997    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerDied","Data":"6bc526085f62762ea4931709dbac708584f1c42a0c84f9a9a955bbdc1e9dd80f"}
Jul 28 01:04:56.554607 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:56.554564    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:56.554774 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:56.554705    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:57.555456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:57.555420    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:57.555918 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:57.555554    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:57.555918 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:57.555603    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:57.555918 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:57.555694    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:58.249224 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:58.249189    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:58.249394 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:58.249257    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:58.249394 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.249359    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:58.249394 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.249368    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:58.249500 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.249423    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:30.249406952 +0000 UTC m=+66.260645541 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 01:04:58.249500 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.249437    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret podName:2c86d027-c224-4ab8-8b80-36d85be6e41b nodeName:}" failed. No retries permitted until 2026-07-28 01:05:30.24943113 +0000 UTC m=+66.260669717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret") pod "global-pull-secret-syncer-hdkfq" (UID: "2c86d027-c224-4ab8-8b80-36d85be6e41b") : object "kube-system"/"original-pull-secret" not registered
Jul 28 01:04:58.349941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:58.349901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:58.350132 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.350070    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 01:04:58.350132 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.350091    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 01:04:58.350132 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.350100    2566 projected.go:196] Error preparing data for projected volume kube-api-access-48xwq for pod openshift-network-diagnostics/network-check-target-v4sm2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:58.350269 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.350168    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq podName:f62c4c73-b44f-42e8-adb5-a77cc281c038 nodeName:}" failed. No retries permitted until 2026-07-28 01:05:30.350137231 +0000 UTC m=+66.361375817 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-48xwq" (UniqueName: "kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq") pod "network-check-target-v4sm2" (UID: "f62c4c73-b44f-42e8-adb5-a77cc281c038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 01:04:58.555337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:58.555258    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:04:58.555499 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:58.555404    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:04:59.555349 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.555316    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:04:59.555349 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.555333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:04:59.555823 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:59.555424    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hdkfq" podUID="2c86d027-c224-4ab8-8b80-36d85be6e41b"
Jul 28 01:04:59.555823 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:04:59.555497    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v4sm2" podUID="f62c4c73-b44f-42e8-adb5-a77cc281c038"
Jul 28 01:04:59.852797 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.852775    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-236.ec2.internal" event="NodeReady"
Jul 28 01:04:59.852937 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.852926    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 01:04:59.860509 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.860485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerStarted","Data":"7edbc0ee830677a0ae2ec776108070e65ea0b42e1d962e25d69d2114cf5bb062"}
Jul 28 01:04:59.900775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.900729    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wvg8g"]
Jul 28 01:04:59.902634 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.902613    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zhqzt"]
Jul 28 01:04:59.902760 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.902736    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:04:59.904706 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.904685    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:04:59.905473 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.905452    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 01:04:59.905581 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.905504    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 01:04:59.905581 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.905568    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9pqrg\""
Jul 28 01:04:59.907041 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.907024    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 01:04:59.907431 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.907408    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-w2j6n\""
Jul 28 01:04:59.907504 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.907436    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 01:04:59.907504 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.907480    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 01:04:59.915253 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.915217    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zhqzt"]
Jul 28 01:04:59.919831 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:04:59.919811    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvg8g"]
Jul 28 01:05:00.063419 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.063384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c966ed32-b941-4d8a-a3aa-449f757957b6-cert\") pod \"ingress-canary-zhqzt\" (UID: \"c966ed32-b941-4d8a-a3aa-449f757957b6\") " pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.063564 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.063465    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctr4q\" (UniqueName: \"kubernetes.io/projected/c966ed32-b941-4d8a-a3aa-449f757957b6-kube-api-access-ctr4q\") pod \"ingress-canary-zhqzt\" (UID: \"c966ed32-b941-4d8a-a3aa-449f757957b6\") " pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.063564 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.063487    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d13e3477-1101-406b-a2a6-4fcd0e206eec-tmp-dir\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.063564 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.063508    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d13e3477-1101-406b-a2a6-4fcd0e206eec-config-volume\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.063564 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.063535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.063702 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.063567    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkgdp\" (UniqueName: \"kubernetes.io/projected/d13e3477-1101-406b-a2a6-4fcd0e206eec-kube-api-access-nkgdp\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.164184 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c966ed32-b941-4d8a-a3aa-449f757957b6-cert\") pod \"ingress-canary-zhqzt\" (UID: \"c966ed32-b941-4d8a-a3aa-449f757957b6\") " pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.164410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctr4q\" (UniqueName: \"kubernetes.io/projected/c966ed32-b941-4d8a-a3aa-449f757957b6-kube-api-access-ctr4q\") pod \"ingress-canary-zhqzt\" (UID: \"c966ed32-b941-4d8a-a3aa-449f757957b6\") " pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.164410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164259    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d13e3477-1101-406b-a2a6-4fcd0e206eec-tmp-dir\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.164410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d13e3477-1101-406b-a2a6-4fcd0e206eec-config-volume\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.164410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.164410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164364    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkgdp\" (UniqueName: \"kubernetes.io/projected/d13e3477-1101-406b-a2a6-4fcd0e206eec-kube-api-access-nkgdp\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.164679 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:00.164510    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:00.164679 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:00.164585    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:05:00.664563885 +0000 UTC m=+36.675802472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:00.164679 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164617    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d13e3477-1101-406b-a2a6-4fcd0e206eec-tmp-dir\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.164870 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.164851    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d13e3477-1101-406b-a2a6-4fcd0e206eec-config-volume\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.168565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.168542    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c966ed32-b941-4d8a-a3aa-449f757957b6-cert\") pod \"ingress-canary-zhqzt\" (UID: \"c966ed32-b941-4d8a-a3aa-449f757957b6\") " pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.174636 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.174614    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkgdp\" (UniqueName: \"kubernetes.io/projected/d13e3477-1101-406b-a2a6-4fcd0e206eec-kube-api-access-nkgdp\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.174773 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.174679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctr4q\" (UniqueName: \"kubernetes.io/projected/c966ed32-b941-4d8a-a3aa-449f757957b6-kube-api-access-ctr4q\") pod \"ingress-canary-zhqzt\" (UID: \"c966ed32-b941-4d8a-a3aa-449f757957b6\") " pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.265854 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.265821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zhqzt"
Jul 28 01:05:00.397549 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.397508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zhqzt"]
Jul 28 01:05:00.402022 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:05:00.401994    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc966ed32_b941_4d8a_a3aa_449f757957b6.slice/crio-2a44602d6a1f2e155e47f6914e23ac3cdcd8e2b838346cce52a1f26c8ec23ac5 WatchSource:0}: Error finding container 2a44602d6a1f2e155e47f6914e23ac3cdcd8e2b838346cce52a1f26c8ec23ac5: Status 404 returned error can't find the container with id 2a44602d6a1f2e155e47f6914e23ac3cdcd8e2b838346cce52a1f26c8ec23ac5
Jul 28 01:05:00.555169 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.555079    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:05:00.558210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.558177    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 01:05:00.558816 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.558177    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qvhk6\""
Jul 28 01:05:00.668076 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.668044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:00.668263 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:00.668195    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:00.668338 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:00.668266    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:05:01.668247418 +0000 UTC m=+37.679486004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:00.866020 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.865749    2566 generic.go:356] "Generic (PLEG): container finished" podID="0fb4f15a-ca09-41c6-86dc-d99584578a2e" containerID="7edbc0ee830677a0ae2ec776108070e65ea0b42e1d962e25d69d2114cf5bb062" exitCode=0
Jul 28 01:05:00.866204 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.865828    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerDied","Data":"7edbc0ee830677a0ae2ec776108070e65ea0b42e1d962e25d69d2114cf5bb062"}
Jul 28 01:05:00.867336 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:00.867312    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zhqzt" event={"ID":"c966ed32-b941-4d8a-a3aa-449f757957b6","Type":"ContainerStarted","Data":"2a44602d6a1f2e155e47f6914e23ac3cdcd8e2b838346cce52a1f26c8ec23ac5"}
Jul 28 01:05:01.555082 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.555045    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:05:01.555251 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.555045    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:05:01.558303 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.558273    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 01:05:01.559139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.558933    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 01:05:01.559139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.558948    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vxkxb\""
Jul 28 01:05:01.559139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.558985    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 01:05:01.674884 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.674851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:01.675072 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:01.675016    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:01.675137 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:01.675091    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:05:03.675072237 +0000 UTC m=+39.686310844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:01.873335 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.873298    2566 generic.go:356] "Generic (PLEG): container finished" podID="0fb4f15a-ca09-41c6-86dc-d99584578a2e" containerID="269caa07e72f24c8083c0c9dc9da9e6641f69dea1429720b0d2b24baf23fcb14" exitCode=0
Jul 28 01:05:01.873515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:01.873356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerDied","Data":"269caa07e72f24c8083c0c9dc9da9e6641f69dea1429720b0d2b24baf23fcb14"}
Jul 28 01:05:02.878986 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:02.878932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" event={"ID":"0fb4f15a-ca09-41c6-86dc-d99584578a2e","Type":"ContainerStarted","Data":"3063f48b2eb03fa7d8959bdb2efa0623e61d0ef21e5ec53879fc38f230a8373c"}
Jul 28 01:05:02.900675 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:02.900603    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rh6sg" podStartSLOduration=8.019241203 podStartE2EDuration="37.900580807s" totalImagesPullingTime="29.881339604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:02.899173012 +0000 UTC m=+38.910411621" watchObservedRunningTime="2026-07-28 01:05:02.900580807 +0000 UTC m=+38.911819419"
Jul 28 01:05:03.691882 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:03.691845    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:03.692058 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:03.691994    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:03.692098 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:03.692058    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:05:07.692042558 +0000 UTC m=+43.703281145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:03.883576 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:03.883535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zhqzt" event={"ID":"c966ed32-b941-4d8a-a3aa-449f757957b6","Type":"ContainerStarted","Data":"6ebbedb3d306762378ffb4427376752d6577028eedc97e52d034fbf49dc711ea"}
Jul 28 01:05:03.897935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:03.897884    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zhqzt" podStartSLOduration=1.5683461730000001 podStartE2EDuration="4.897865811s" totalImagesPullingTime="3.329519638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:03.897407567 +0000 UTC m=+39.908646177" watchObservedRunningTime="2026-07-28 01:05:03.897865811 +0000 UTC m=+39.909104419"
Jul 28 01:05:07.722389 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:07.722196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:07.722887 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:07.722357    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:07.722887 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:07.722491    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:05:15.722470332 +0000 UTC m=+51.733708931 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:15.777052 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:15.777005    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:15.777538 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:15.777148    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:15.777538 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:15.777213    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:05:31.777195802 +0000 UTC m=+67.788434389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:25.866839 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:25.866809    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j49pf"
Jul 28 01:05:30.280221 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.280172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:05:30.280221 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.280226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:05:30.283006 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.282963    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 01:05:30.283067 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.283012    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 01:05:30.291278 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:30.291242    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 01:05:30.291364 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:30.291342    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:06:34.291316559 +0000 UTC m=+130.302555167 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : secret "metrics-daemon-secret" not found
Jul 28 01:05:30.295198 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.295176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2c86d027-c224-4ab8-8b80-36d85be6e41b-original-pull-secret\") pod \"global-pull-secret-syncer-hdkfq\" (UID: \"2c86d027-c224-4ab8-8b80-36d85be6e41b\") " pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:05:30.367777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.367737    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hdkfq"
Jul 28 01:05:30.380935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.380902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:05:30.383728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.383702    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 01:05:30.393735 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.393708    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 01:05:30.405156 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.405128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48xwq\" (UniqueName: \"kubernetes.io/projected/f62c4c73-b44f-42e8-adb5-a77cc281c038-kube-api-access-48xwq\") pod \"network-check-target-v4sm2\" (UID: \"f62c4c73-b44f-42e8-adb5-a77cc281c038\") " pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:05:30.492837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.492803    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hdkfq"]
Jul 28 01:05:30.496163 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:05:30.496131    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c86d027_c224_4ab8_8b80_36d85be6e41b.slice/crio-72c8add70573e990af0d8e98b1362589c3f7ba52cb2edc303afc0bebc9c6852a WatchSource:0}: Error finding container 72c8add70573e990af0d8e98b1362589c3f7ba52cb2edc303afc0bebc9c6852a: Status 404 returned error can't find the container with id 72c8add70573e990af0d8e98b1362589c3f7ba52cb2edc303afc0bebc9c6852a
Jul 28 01:05:30.675792 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.675756    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vxkxb\""
Jul 28 01:05:30.682993 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.682953    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:05:30.831167 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.831139    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v4sm2"]
Jul 28 01:05:30.834240 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:05:30.834211    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf62c4c73_b44f_42e8_adb5_a77cc281c038.slice/crio-a10a3a0095fe5ba54ae4e85886f90afc72f1b913881690b1a7168f54ad3f9e9c WatchSource:0}: Error finding container a10a3a0095fe5ba54ae4e85886f90afc72f1b913881690b1a7168f54ad3f9e9c: Status 404 returned error can't find the container with id a10a3a0095fe5ba54ae4e85886f90afc72f1b913881690b1a7168f54ad3f9e9c
Jul 28 01:05:30.944600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.944513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hdkfq" event={"ID":"2c86d027-c224-4ab8-8b80-36d85be6e41b","Type":"ContainerStarted","Data":"72c8add70573e990af0d8e98b1362589c3f7ba52cb2edc303afc0bebc9c6852a"}
Jul 28 01:05:30.945772 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:30.945714    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v4sm2" event={"ID":"f62c4c73-b44f-42e8-adb5-a77cc281c038","Type":"ContainerStarted","Data":"a10a3a0095fe5ba54ae4e85886f90afc72f1b913881690b1a7168f54ad3f9e9c"}
Jul 28 01:05:31.791894 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:31.791855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:05:31.792457 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:31.792051    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:05:31.792457 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:05:31.792135    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:06:03.792113616 +0000 UTC m=+99.803352203 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:05:35.960561 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:35.960519    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v4sm2" event={"ID":"f62c4c73-b44f-42e8-adb5-a77cc281c038","Type":"ContainerStarted","Data":"ad93a07b0cb8d103af9608d495a262858471e570820f55114c6f6ef7f78b81c4"}
Jul 28 01:05:35.960937 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:35.960641    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:05:35.962180 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:35.962158    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hdkfq" event={"ID":"2c86d027-c224-4ab8-8b80-36d85be6e41b","Type":"ContainerStarted","Data":"8a63c07cb04fa766d86d9f4a18e7137c3aaa80cb9e039a58de8d0ce421f0c56b"}
Jul 28 01:05:35.974102 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:35.974061    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v4sm2" podStartSLOduration=66.540703958 podStartE2EDuration="1m10.97404788s" totalImagesPullingTime="4.433343922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:35.973733771 +0000 UTC m=+71.984972380" watchObservedRunningTime="2026-07-28 01:05:35.97404788 +0000 UTC m=+71.985286485"
Jul 28 01:05:35.987016 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:35.986953    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hdkfq" podStartSLOduration=66.213236131 podStartE2EDuration="1m10.986940688s" totalImagesPullingTime="4.773704557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:35.986335218 +0000 UTC m=+71.997573828" watchObservedRunningTime="2026-07-28 01:05:35.986940688 +0000 UTC m=+71.998179296"
Jul 28 01:05:43.190666 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.190629    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"]
Jul 28 01:05:43.192759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.192738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.195957 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.195933    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 01:05:43.197064 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.197042    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 01:05:43.197312 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.197298    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 01:05:43.197374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.197295    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 01:05:43.208110 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.208080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"]
Jul 28 01:05:43.269599 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.269561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/726ac681-c200-411b-82de-4e7a59ac44f3-klusterlet-config\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.269599 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.269600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/726ac681-c200-411b-82de-4e7a59ac44f3-tmp\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.269831 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.269623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mvhk\" (UniqueName: \"kubernetes.io/projected/726ac681-c200-411b-82de-4e7a59ac44f3-kube-api-access-8mvhk\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.370214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.370171    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/726ac681-c200-411b-82de-4e7a59ac44f3-tmp\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.370214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.370214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mvhk\" (UniqueName: \"kubernetes.io/projected/726ac681-c200-411b-82de-4e7a59ac44f3-kube-api-access-8mvhk\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.370317 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.370265    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/726ac681-c200-411b-82de-4e7a59ac44f3-klusterlet-config\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.370566 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.370542    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/726ac681-c200-411b-82de-4e7a59ac44f3-tmp\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.372730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.372713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/726ac681-c200-411b-82de-4e7a59ac44f3-klusterlet-config\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.378762 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.378737    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mvhk\" (UniqueName: \"kubernetes.io/projected/726ac681-c200-411b-82de-4e7a59ac44f3-kube-api-access-8mvhk\") pod \"klusterlet-addon-workmgr-75d55847bb-2l4g8\" (UID: \"726ac681-c200-411b-82de-4e7a59ac44f3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.501988 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.501883    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:43.622475 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.622446    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"]
Jul 28 01:05:43.625898 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:05:43.625863    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726ac681_c200_411b_82de_4e7a59ac44f3.slice/crio-4ada56cb73b3c07ca56a91affebc5fdf067d7b5e698ca7aeb7a4758e7472da80 WatchSource:0}: Error finding container 4ada56cb73b3c07ca56a91affebc5fdf067d7b5e698ca7aeb7a4758e7472da80: Status 404 returned error can't find the container with id 4ada56cb73b3c07ca56a91affebc5fdf067d7b5e698ca7aeb7a4758e7472da80
Jul 28 01:05:43.984530 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:43.984493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8" event={"ID":"726ac681-c200-411b-82de-4e7a59ac44f3","Type":"ContainerStarted","Data":"4ada56cb73b3c07ca56a91affebc5fdf067d7b5e698ca7aeb7a4758e7472da80"}
Jul 28 01:05:48.998810 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:48.998780    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8" event={"ID":"726ac681-c200-411b-82de-4e7a59ac44f3","Type":"ContainerStarted","Data":"00f6cbbd6ec3a3cb8b10cb18a45a0c25d9f34bb2eecfb259d99f3526d06a211d"}
Jul 28 01:05:48.999221 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:48.999011    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:49.002911 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:49.002886    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:05:49.014867 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:05:49.014802    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8" podStartSLOduration=1.668730791 podStartE2EDuration="6.014783374s" totalImagesPullingTime="4.346052583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:05:49.014175855 +0000 UTC m=+85.025414472" watchObservedRunningTime="2026-07-28 01:05:49.014783374 +0000 UTC m=+85.026021988"
Jul 28 01:06:03.818863 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:06:03.818817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:06:03.819318 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:06:03.818999    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:06:03.819318 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:06:03.819066    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:07:07.81904777 +0000 UTC m=+163.830286357 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:06:06.967175 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:06:06.967143    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v4sm2"
Jul 28 01:06:34.322741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:06:34.322690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:06:34.323237 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:06:34.322852    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 28 01:06:34.323237 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:06:34.322921    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs podName:b15ae3bf-ee89-4a3f-9ae1-565c670d2b31 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:36.322903322 +0000 UTC m=+252.334141910 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs") pod "network-metrics-daemon-fnd2j" (UID: "b15ae3bf-ee89-4a3f-9ae1-565c670d2b31") : secret "metrics-daemon-secret" not found
Jul 28 01:07:02.914472 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:07:02.914422    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wvg8g" podUID="d13e3477-1101-406b-a2a6-4fcd0e206eec"
Jul 28 01:07:03.172189 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:03.172107    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:07:03.564541 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:07:03.564444    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-fnd2j" podUID="b15ae3bf-ee89-4a3f-9ae1-565c670d2b31"
Jul 28 01:07:07.851356 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:07.851294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:07:07.851871 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:07:07.851451    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 01:07:07.851871 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:07:07.851543    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls podName:d13e3477-1101-406b-a2a6-4fcd0e206eec nodeName:}" failed. No retries permitted until 2026-07-28 01:09:09.851518174 +0000 UTC m=+285.862756764 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls") pod "dns-default-wvg8g" (UID: "d13e3477-1101-406b-a2a6-4fcd0e206eec") : secret "dns-default-metrics-tls" not found
Jul 28 01:07:14.556392 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:14.556294    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:07:48.280889 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:48.280850    2566 generic.go:356] "Generic (PLEG): container finished" podID="726ac681-c200-411b-82de-4e7a59ac44f3" containerID="00f6cbbd6ec3a3cb8b10cb18a45a0c25d9f34bb2eecfb259d99f3526d06a211d" exitCode=1
Jul 28 01:07:48.281315 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:48.280905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8" event={"ID":"726ac681-c200-411b-82de-4e7a59ac44f3","Type":"ContainerDied","Data":"00f6cbbd6ec3a3cb8b10cb18a45a0c25d9f34bb2eecfb259d99f3526d06a211d"}
Jul 28 01:07:48.281315 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:48.281290    2566 scope.go:117] "RemoveContainer" containerID="00f6cbbd6ec3a3cb8b10cb18a45a0c25d9f34bb2eecfb259d99f3526d06a211d"
Jul 28 01:07:48.999355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:48.999309    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:07:49.285315 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:49.285232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8" event={"ID":"726ac681-c200-411b-82de-4e7a59ac44f3","Type":"ContainerStarted","Data":"19faa9d447506bc25fd10c6ca98fee08f756b2351a32b6353c32c3548d92bd78"}
Jul 28 01:07:49.285698 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:49.285449    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:07:49.286043 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:07:49.286017    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-75d55847bb-2l4g8"
Jul 28 01:08:10.076782 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.076742    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"]
Jul 28 01:08:10.079902 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.079884    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"]
Jul 28 01:08:10.080053 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.080035    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.082558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.082539    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55bddfb586-4qmrj"]
Jul 28 01:08:10.082691 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.082671    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"
Jul 28 01:08:10.083221 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.083204    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-wp2g4\""
Jul 28 01:08:10.083325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.083259    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.083476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.083454    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 01:08:10.083586 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.083511    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.083586 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.083522    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 01:08:10.085385 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.085364    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-kbxgx\""
Jul 28 01:08:10.085745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.085719    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.085745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.085734    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.085898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.085816    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.089384 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.089364    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh4w\""
Jul 28 01:08:10.089720 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.089702    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 01:08:10.089720 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.089717    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 01:08:10.089881 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.089759    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 01:08:10.092324 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.092299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"]
Jul 28 01:08:10.094858 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.094835    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 01:08:10.099547 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.099528    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55bddfb586-4qmrj"]
Jul 28 01:08:10.102479 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.102455    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"]
Jul 28 01:08:10.151220 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151181    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-image-registry-private-configuration\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151220 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151218    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-certificates\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151250    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/106ccc7e-fbc5-4b90-8eee-829b1146ed20-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.151472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/106ccc7e-fbc5-4b90-8eee-829b1146ed20-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.151472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151369    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qflbn\" (UniqueName: \"kubernetes.io/projected/cb54c5d5-d5ac-4456-95c6-c1d8de186d47-kube-api-access-qflbn\") pod \"volume-data-source-validator-549c6d54cf-zp9pj\" (UID: \"cb54c5d5-d5ac-4456-95c6-c1d8de186d47\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"
Jul 28 01:08:10.151472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151385    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-bound-sa-token\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-installation-pull-secrets\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nz7\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-kube-api-access-m4nz7\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151587    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-trusted-ca\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.151743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151605    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njrf6\" (UniqueName: \"kubernetes.io/projected/106ccc7e-fbc5-4b90-8eee-829b1146ed20-kube-api-access-njrf6\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.151743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.151624    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-ca-trust-extracted\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.171987 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.171890    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"]
Jul 28 01:08:10.175343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.175316    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m68h5"]
Jul 28 01:08:10.175449 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.175432    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.177723 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.177703    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.177924 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.177909    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 01:08:10.178047 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.177950    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.178047 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.177952    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5x67p\""
Jul 28 01:08:10.178702 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.178674    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5b598fd8c5-s7dmr"]
Jul 28 01:08:10.178809 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.178789    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.181063 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.181043    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.181169 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.181073    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 01:08:10.181169 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.181046    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-sbvj9\""
Jul 28 01:08:10.181331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.181315    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 01:08:10.181630 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.181612    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.182189 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.182171    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.184760 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.184736    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"]
Jul 28 01:08:10.185397 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185377    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 01:08:10.185540 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185516    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 01:08:10.185649 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185399    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 01:08:10.185649 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185434    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vps7m\""
Jul 28 01:08:10.185759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185439    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 01:08:10.185759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185476    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 01:08:10.185759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185485    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 01:08:10.185895 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.185836    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m68h5"]
Jul 28 01:08:10.186535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.186513    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 01:08:10.191153 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.190749    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b598fd8c5-s7dmr"]
Jul 28 01:08:10.252461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252424    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-image-registry-private-configuration\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.252461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252465    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/106ccc7e-fbc5-4b90-8eee-829b1146ed20-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.252728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/106ccc7e-fbc5-4b90-8eee-829b1146ed20-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.252728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252610    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qflbn\" (UniqueName: \"kubernetes.io/projected/cb54c5d5-d5ac-4456-95c6-c1d8de186d47-kube-api-access-qflbn\") pod \"volume-data-source-validator-549c6d54cf-zp9pj\" (UID: \"cb54c5d5-d5ac-4456-95c6-c1d8de186d47\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"
Jul 28 01:08:10.252728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.252728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzbhd\" (UniqueName: \"kubernetes.io/projected/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-kube-api-access-rzbhd\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-serving-cert\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252804    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nz7\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-kube-api-access-m4nz7\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-service-ca-bundle\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-kube-api-access-vcpqq\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njrf6\" (UniqueName: \"kubernetes.io/projected/106ccc7e-fbc5-4b90-8eee-829b1146ed20-kube-api-access-njrf6\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-ca-trust-extracted\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.252941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252925    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-default-certificate\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.252952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-tmp\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253095    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-certificates\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253170    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/106ccc7e-fbc5-4b90-8eee-829b1146ed20-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-snapshots\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-bound-sa-token\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253243    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-stats-auth\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253268    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5fjk\" (UniqueName: \"kubernetes.io/projected/7a320d79-4cb8-439d-8827-32318d0987a1-kube-api-access-t5fjk\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.253284    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.253300    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bddfb586-4qmrj: secret "image-registry-tls" not found
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253282    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-ca-trust-extracted\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.253374 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.253379    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls podName:bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:10.75336555 +0000 UTC m=+226.764604136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls") pod "image-registry-55bddfb586-4qmrj" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7") : secret "image-registry-tls" not found
Jul 28 01:08:10.253898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253404    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-installation-pull-secrets\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.253898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253438    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-trusted-ca\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.253898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.253898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.253838    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-certificates\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.254297 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.254280    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-trusted-ca\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.255413 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.255392    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/106ccc7e-fbc5-4b90-8eee-829b1146ed20-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.255561 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.255544    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-image-registry-private-configuration\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.255787 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.255768    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-installation-pull-secrets\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.263954 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.263932    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qflbn\" (UniqueName: \"kubernetes.io/projected/cb54c5d5-d5ac-4456-95c6-c1d8de186d47-kube-api-access-qflbn\") pod \"volume-data-source-validator-549c6d54cf-zp9pj\" (UID: \"cb54c5d5-d5ac-4456-95c6-c1d8de186d47\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"
Jul 28 01:08:10.267777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.267747    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njrf6\" (UniqueName: \"kubernetes.io/projected/106ccc7e-fbc5-4b90-8eee-829b1146ed20-kube-api-access-njrf6\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-6764s\" (UID: \"106ccc7e-fbc5-4b90-8eee-829b1146ed20\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.270674 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.270645    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-bound-sa-token\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.271037 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.271016    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nz7\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-kube-api-access-m4nz7\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.353927 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.353889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.353942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-snapshots\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354047    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-stats-auth\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.354115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5fjk\" (UniqueName: \"kubernetes.io/projected/7a320d79-4cb8-439d-8827-32318d0987a1-kube-api-access-t5fjk\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.354261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.354261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354163    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.354261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzbhd\" (UniqueName: \"kubernetes.io/projected/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-kube-api-access-rzbhd\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-serving-cert\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-service-ca-bundle\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-kube-api-access-vcpqq\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.354285    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354309    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-default-certificate\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.354352    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:10.854331494 +0000 UTC m=+226.865570106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : secret "router-metrics-certs-default" not found
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354371    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-tmp\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354397    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.354445    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:10.854429193 +0000 UTC m=+226.865667798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : configmap references non-existent config key: service-ca.crt
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.354474    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 01:08:10.354545 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.354510    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls podName:7a320d79-4cb8-439d-8827-32318d0987a1 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:10.854498041 +0000 UTC m=+226.865736631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg4t5" (UID: "7a320d79-4cb8-439d-8827-32318d0987a1") : secret "samples-operator-tls" not found
Jul 28 01:08:10.354998 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-snapshots\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.354998 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.354870    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-tmp\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.355135 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.355094    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-service-ca-bundle\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.355745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.355687    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.356797 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.356775    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-serving-cert\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.356797 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.356785    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-default-certificate\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.356942 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.356920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-stats-auth\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.362287 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.362268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzbhd\" (UniqueName: \"kubernetes.io/projected/253a805e-0ba8-4c8d-ab32-776f7eeb93ab-kube-api-access-rzbhd\") pod \"insights-operator-5885fdb759-m68h5\" (UID: \"253a805e-0ba8-4c8d-ab32-776f7eeb93ab\") " pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.362367 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.362355    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5fjk\" (UniqueName: \"kubernetes.io/projected/7a320d79-4cb8-439d-8827-32318d0987a1-kube-api-access-t5fjk\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.362525 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.362508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcpqq\" (UniqueName: \"kubernetes.io/projected/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-kube-api-access-vcpqq\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.389825 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.389790    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"
Jul 28 01:08:10.398154 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.398111    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"
Jul 28 01:08:10.494124 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.493916    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-m68h5"
Jul 28 01:08:10.521533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.521502    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s"]
Jul 28 01:08:10.524419 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:10.524378    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod106ccc7e_fbc5_4b90_8eee_829b1146ed20.slice/crio-5b9b9ce0e3493628b36e53022301a1a7ef7f328b46c7c9549634ade318535c5d WatchSource:0}: Error finding container 5b9b9ce0e3493628b36e53022301a1a7ef7f328b46c7c9549634ade318535c5d: Status 404 returned error can't find the container with id 5b9b9ce0e3493628b36e53022301a1a7ef7f328b46c7c9549634ade318535c5d
Jul 28 01:08:10.544072 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.544039    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj"]
Jul 28 01:08:10.547594 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:10.547560    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb54c5d5_d5ac_4456_95c6_c1d8de186d47.slice/crio-4b188667af2a3d8d7e726bf5ba9c79cc3cb793249c35ec372d4bc103d8b965ea WatchSource:0}: Error finding container 4b188667af2a3d8d7e726bf5ba9c79cc3cb793249c35ec372d4bc103d8b965ea: Status 404 returned error can't find the container with id 4b188667af2a3d8d7e726bf5ba9c79cc3cb793249c35ec372d4bc103d8b965ea
Jul 28 01:08:10.621407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.621351    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-m68h5"]
Jul 28 01:08:10.623760 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:10.623718    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod253a805e_0ba8_4c8d_ab32_776f7eeb93ab.slice/crio-99e7bbea28a7171e41d3bd0dfcabb8a078cd368521bc2e8f1b1e034820bd8058 WatchSource:0}: Error finding container 99e7bbea28a7171e41d3bd0dfcabb8a078cd368521bc2e8f1b1e034820bd8058: Status 404 returned error can't find the container with id 99e7bbea28a7171e41d3bd0dfcabb8a078cd368521bc2e8f1b1e034820bd8058
Jul 28 01:08:10.757882 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.757852    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:10.758059 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.757951    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:08:10.758059 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.757963    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bddfb586-4qmrj: secret "image-registry-tls" not found
Jul 28 01:08:10.758059 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.758031    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls podName:bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:11.75801743 +0000 UTC m=+227.769256019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls") pod "image-registry-55bddfb586-4qmrj" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7") : secret "image-registry-tls" not found
Jul 28 01:08:10.858302 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.858267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.858468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.858313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:10.858468 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.858407    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 01:08:10.858468 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.858406    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 01:08:10.858468 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.858470    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:11.858454124 +0000 UTC m=+227.869692712 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : secret "router-metrics-certs-default" not found
Jul 28 01:08:10.858600 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.858483    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls podName:7a320d79-4cb8-439d-8827-32318d0987a1 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:11.858477505 +0000 UTC m=+227.869716092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg4t5" (UID: "7a320d79-4cb8-439d-8827-32318d0987a1") : secret "samples-operator-tls" not found
Jul 28 01:08:10.858600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:10.858527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:10.858665 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:10.858612    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:11.858604964 +0000 UTC m=+227.869843550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : configmap references non-existent config key: service-ca.crt
Jul 28 01:08:11.340166 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.340124    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s" event={"ID":"106ccc7e-fbc5-4b90-8eee-829b1146ed20","Type":"ContainerStarted","Data":"5b9b9ce0e3493628b36e53022301a1a7ef7f328b46c7c9549634ade318535c5d"}
Jul 28 01:08:11.341765 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.341734    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m68h5" event={"ID":"253a805e-0ba8-4c8d-ab32-776f7eeb93ab","Type":"ContainerStarted","Data":"99e7bbea28a7171e41d3bd0dfcabb8a078cd368521bc2e8f1b1e034820bd8058"}
Jul 28 01:08:11.343865 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.343831    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj" event={"ID":"cb54c5d5-d5ac-4456-95c6-c1d8de186d47","Type":"ContainerStarted","Data":"4b188667af2a3d8d7e726bf5ba9c79cc3cb793249c35ec372d4bc103d8b965ea"}
Jul 28 01:08:11.768199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.768117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:11.768362 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.768268    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:08:11.768362 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.768286    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bddfb586-4qmrj: secret "image-registry-tls" not found
Jul 28 01:08:11.768362 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.768349    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls podName:bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:13.768333693 +0000 UTC m=+229.779572295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls") pod "image-registry-55bddfb586-4qmrj" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7") : secret "image-registry-tls" not found
Jul 28 01:08:11.868749 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.868715    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:11.868936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.868774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:11.868936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:11.868815    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:11.868936 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.868912    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:13.868887732 +0000 UTC m=+229.880126331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : configmap references non-existent config key: service-ca.crt
Jul 28 01:08:11.869178 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.868987    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 01:08:11.869178 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.869040    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls podName:7a320d79-4cb8-439d-8827-32318d0987a1 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:13.869024092 +0000 UTC m=+229.880262681 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg4t5" (UID: "7a320d79-4cb8-439d-8827-32318d0987a1") : secret "samples-operator-tls" not found
Jul 28 01:08:11.869178 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.869104    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 01:08:11.869178 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:11.869142    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:13.869131449 +0000 UTC m=+229.880370037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : secret "router-metrics-certs-default" not found
Jul 28 01:08:12.353807 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:12.353757    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj" event={"ID":"cb54c5d5-d5ac-4456-95c6-c1d8de186d47","Type":"ContainerStarted","Data":"01de808e2029491e71ab2b221352373f67a3ebdaf2383c7dee9c4418d08dafe5"}
Jul 28 01:08:12.368181 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:12.368130    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zp9pj" podStartSLOduration=1.005071746 podStartE2EDuration="2.368111522s" totalImagesPullingTime="1.363039776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:12.367000192 +0000 UTC m=+228.378238800" watchObservedRunningTime="2026-07-28 01:08:12.368111522 +0000 UTC m=+228.379350132"
Jul 28 01:08:13.359410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.359367    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s" event={"ID":"106ccc7e-fbc5-4b90-8eee-829b1146ed20","Type":"ContainerStarted","Data":"81b6834af5cadf4751d18bc6c70dca4d7d6e457b33f24ee7fd5bf500564e60db"}
Jul 28 01:08:13.360915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.360877    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m68h5" event={"ID":"253a805e-0ba8-4c8d-ab32-776f7eeb93ab","Type":"ContainerStarted","Data":"3bfcc03f35c8bac50cfdbab68c08c98c4b4c8531666c78a10cbb6153fd575841"}
Jul 28 01:08:13.376089 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.376034    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s" podStartSLOduration=0.744518039 podStartE2EDuration="3.376015918s" totalImagesPullingTime="2.631497879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:13.373991406 +0000 UTC m=+229.385230017" watchObservedRunningTime="2026-07-28 01:08:13.376015918 +0000 UTC m=+229.387254528"
Jul 28 01:08:13.390322 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.390268    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-m68h5" podStartSLOduration=0.85521738 podStartE2EDuration="3.390247228s" totalImagesPullingTime="2.535029848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:13.389009648 +0000 UTC m=+229.400248259" watchObservedRunningTime="2026-07-28 01:08:13.390247228 +0000 UTC m=+229.401485839"
Jul 28 01:08:13.785885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.785845    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:13.786091 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.786030    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:08:13.786091 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.786053    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bddfb586-4qmrj: secret "image-registry-tls" not found
Jul 28 01:08:13.786173 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.786124    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls podName:bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:17.786101427 +0000 UTC m=+233.797340034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls") pod "image-registry-55bddfb586-4qmrj" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7") : secret "image-registry-tls" not found
Jul 28 01:08:13.886523 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.886468    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:13.886686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.886597    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:13.886686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:13.886641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:13.886686 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.886644    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 01:08:13.886835 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.886728    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls podName:7a320d79-4cb8-439d-8827-32318d0987a1 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:17.886707225 +0000 UTC m=+233.897945817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg4t5" (UID: "7a320d79-4cb8-439d-8827-32318d0987a1") : secret "samples-operator-tls" not found
Jul 28 01:08:13.886835 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.886748    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:17.886734203 +0000 UTC m=+233.897972789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : configmap references non-existent config key: service-ca.crt
Jul 28 01:08:13.886835 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.886760    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 01:08:13.886835 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:13.886809    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:17.886796549 +0000 UTC m=+233.898035146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : secret "router-metrics-certs-default" not found
Jul 28 01:08:14.535858 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.535823    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cd6hz"]
Jul 28 01:08:14.539642 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.539586    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.541625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.541597    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7gpmd\""
Jul 28 01:08:14.541786 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.541764    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 01:08:14.541831 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.541796    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 01:08:14.549805 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.549788    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cd6hz"]
Jul 28 01:08:14.591564 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.591532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c92r\" (UniqueName: \"kubernetes.io/projected/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-api-access-4c92r\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.591732 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.591610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.591732 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.591641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/197c23c3-1540-404a-91ff-5a62e4fa333f-crio-socket\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.591827 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.591728    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.591827 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.591755    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/197c23c3-1540-404a-91ff-5a62e4fa333f-data-volume\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692141 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692141 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/197c23c3-1540-404a-91ff-5a62e4fa333f-data-volume\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692427 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:14.692238    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:14.692427 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692269    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c92r\" (UniqueName: \"kubernetes.io/projected/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-api-access-4c92r\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692427 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:14.692315    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:15.192291661 +0000 UTC m=+231.203530247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:14.692427 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692427 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692417    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/197c23c3-1540-404a-91ff-5a62e4fa333f-crio-socket\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692671 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:14.692527    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:14.692671 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:14.692592    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:15.192574908 +0000 UTC m=+231.203813514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:14.692671 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692530    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/197c23c3-1540-404a-91ff-5a62e4fa333f-crio-socket\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.692671 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.692640    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/197c23c3-1540-404a-91ff-5a62e4fa333f-data-volume\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:14.700203 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:14.700179    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c92r\" (UniqueName: \"kubernetes.io/projected/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-api-access-4c92r\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:15.195664 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:15.195626    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:15.195846 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:15.195715    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:15.195890 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:15.195852    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:15.199522 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:15.196191    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:15.199522 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:15.196211    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:16.196185682 +0000 UTC m=+232.207424282 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:15.199522 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:15.196268    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:16.196237458 +0000 UTC m=+232.207476045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:16.204581 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:16.204539    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:16.204999 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:16.204622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:16.204999 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:16.204673    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:16.204999 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:16.204731    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:16.204999 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:16.204750    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:18.20473432 +0000 UTC m=+234.215972907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:16.204999 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:16.204779    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:18.204767863 +0000 UTC m=+234.216006451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:17.128805 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:17.128777    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-58nwp_db9c66a8-6859-4e59-a9ec-938b424484f0/dns-node-resolver/0.log"
Jul 28 01:08:17.816916 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:17.816879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:17.817343 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.817050    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 01:08:17.817343 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.817070    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55bddfb586-4qmrj: secret "image-registry-tls" not found
Jul 28 01:08:17.817343 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.817132    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls podName:bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:25.817117743 +0000 UTC m=+241.828356331 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls") pod "image-registry-55bddfb586-4qmrj" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7") : secret "image-registry-tls" not found
Jul 28 01:08:17.917916 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:17.917883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:17.918057 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:17.917986    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:17.918057 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:17.918016    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:17.918125 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.918084    2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 01:08:17.918125 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.918115    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 01:08:17.918183 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.918159    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:25.918139036 +0000 UTC m=+241.929377634 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : configmap references non-existent config key: service-ca.crt
Jul 28 01:08:17.918222 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.918182    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls podName:7a320d79-4cb8-439d-8827-32318d0987a1 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:25.918174103 +0000 UTC m=+241.929412689 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls") pod "cluster-samples-operator-6748489656-vg4t5" (UID: "7a320d79-4cb8-439d-8827-32318d0987a1") : secret "samples-operator-tls" not found
Jul 28 01:08:17.918222 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:17.918197    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs podName:2e916bb7-6c9e-40e5-9ca7-a7d242bd9384 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:25.91818911 +0000 UTC m=+241.929427714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs") pod "router-default-5b598fd8c5-s7dmr" (UID: "2e916bb7-6c9e-40e5-9ca7-a7d242bd9384") : secret "router-metrics-certs-default" not found
Jul 28 01:08:18.220224 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:18.220185    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:18.220413 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:18.220276    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:18.220413 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:18.220329    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:18.220413 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:18.220366    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:18.220413 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:18.220414    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:22.220392011 +0000 UTC m=+238.231630601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:18.220567 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:18.220429    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:22.220423153 +0000 UTC m=+238.231661740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:18.529571 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:18.529500    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n6jdz_8443e39b-0f3d-423c-ad49-0ab06a0140e8/node-ca/0.log"
Jul 28 01:08:19.329001 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:19.328961    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zhqzt_c966ed32-b941-4d8a-a3aa-449f757957b6/serve-healthcheck-canary/0.log"
Jul 28 01:08:19.930353 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:19.930317    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-6764s_106ccc7e-fbc5-4b90-8eee-829b1146ed20/kube-storage-version-migrator-operator/0.log"
Jul 28 01:08:22.254304 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:22.254262    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:22.254746 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:22.254352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:22.254746 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:22.254403    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:22.254746 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:22.254441    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 01:08:22.254746 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:22.254461    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:30.254446074 +0000 UTC m=+246.265684660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : secret "insights-runtime-extractor-tls" not found
Jul 28 01:08:22.254746 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:22.254495    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm podName:197c23c3-1540-404a-91ff-5a62e4fa333f nodeName:}" failed. No retries permitted until 2026-07-28 01:08:30.254480299 +0000 UTC m=+246.265718886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cd6hz" (UID: "197c23c3-1540-404a-91ff-5a62e4fa333f") : configmap "kube-rbac-proxy" not found
Jul 28 01:08:25.886692 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.886651    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:25.889157 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.889133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"image-registry-55bddfb586-4qmrj\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") " pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:25.987821 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.987778    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:25.988017 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.987863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:25.988017 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.987892    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:25.988616 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.988596    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-service-ca-bundle\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:25.990384 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.990354    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e916bb7-6c9e-40e5-9ca7-a7d242bd9384-metrics-certs\") pod \"router-default-5b598fd8c5-s7dmr\" (UID: \"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384\") " pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:25.990486 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:25.990454    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a320d79-4cb8-439d-8827-32318d0987a1-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vg4t5\" (UID: \"7a320d79-4cb8-439d-8827-32318d0987a1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:26.007074 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.007047    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bzh4w\""
Jul 28 01:08:26.015076 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.015053    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:26.089088 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.089062    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5x67p\""
Jul 28 01:08:26.097210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.097187    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"
Jul 28 01:08:26.100808 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.100782    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-vps7m\""
Jul 28 01:08:26.108810 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.108787    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:26.147076 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.146917    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55bddfb586-4qmrj"]
Jul 28 01:08:26.152136 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:26.152106    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbb7d30_ccc4_4e34_9bbc_e620e1a91ce7.slice/crio-e3d5319179850495ab36c217fd5ce1d1757ff2bd363582b04b6ada1e00183e50 WatchSource:0}: Error finding container e3d5319179850495ab36c217fd5ce1d1757ff2bd363582b04b6ada1e00183e50: Status 404 returned error can't find the container with id e3d5319179850495ab36c217fd5ce1d1757ff2bd363582b04b6ada1e00183e50
Jul 28 01:08:26.234493 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.234460    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5"]
Jul 28 01:08:26.257883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.257713    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5b598fd8c5-s7dmr"]
Jul 28 01:08:26.261592 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:26.261562    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e916bb7_6c9e_40e5_9ca7_a7d242bd9384.slice/crio-9082c88eff74c35920c55b4349fd621e888dadadecbda790991bdc4cad7a1052 WatchSource:0}: Error finding container 9082c88eff74c35920c55b4349fd621e888dadadecbda790991bdc4cad7a1052: Status 404 returned error can't find the container with id 9082c88eff74c35920c55b4349fd621e888dadadecbda790991bdc4cad7a1052
Jul 28 01:08:26.398871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.398776    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr" event={"ID":"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384","Type":"ContainerStarted","Data":"348775a06cc492916d49f1ff66b0b120951c6f1a77074b086635b851eb52a6bb"}
Jul 28 01:08:26.398871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.398836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr" event={"ID":"2e916bb7-6c9e-40e5-9ca7-a7d242bd9384","Type":"ContainerStarted","Data":"9082c88eff74c35920c55b4349fd621e888dadadecbda790991bdc4cad7a1052"}
Jul 28 01:08:26.400001 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.399955    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5" event={"ID":"7a320d79-4cb8-439d-8827-32318d0987a1","Type":"ContainerStarted","Data":"c8c96b7b8cb7a54695be86f86aa9741d6906de628034a9c419a351d55d2cf1f4"}
Jul 28 01:08:26.401535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.401513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" event={"ID":"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7","Type":"ContainerStarted","Data":"690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f"}
Jul 28 01:08:26.401625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.401544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" event={"ID":"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7","Type":"ContainerStarted","Data":"e3d5319179850495ab36c217fd5ce1d1757ff2bd363582b04b6ada1e00183e50"}
Jul 28 01:08:26.401667 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.401633    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:08:26.415856 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.415804    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr" podStartSLOduration=16.415783171 podStartE2EDuration="16.415783171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:26.415424654 +0000 UTC m=+242.426663264" watchObservedRunningTime="2026-07-28 01:08:26.415783171 +0000 UTC m=+242.427021781"
Jul 28 01:08:26.435995 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:26.434330    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" podStartSLOduration=16.434309996 podStartE2EDuration="16.434309996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:26.432539703 +0000 UTC m=+242.443778307" watchObservedRunningTime="2026-07-28 01:08:26.434309996 +0000 UTC m=+242.445548606"
Jul 28 01:08:27.109539 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:27.109504    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:27.112238 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:27.112215    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:27.405584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:27.405502    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:27.406851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:27.406830    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5b598fd8c5-s7dmr"
Jul 28 01:08:28.410119 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:28.410079    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5" event={"ID":"7a320d79-4cb8-439d-8827-32318d0987a1","Type":"ContainerStarted","Data":"243db7f6a0f47934084d108678b3949f4eb8c55d6e89d5a406b457df53caa4db"}
Jul 28 01:08:28.410523 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:28.410131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5" event={"ID":"7a320d79-4cb8-439d-8827-32318d0987a1","Type":"ContainerStarted","Data":"b7be18afdf068cf3674cee1bca0bd14592e82a32f637d66b97147703c6e4dd02"}
Jul 28 01:08:28.427701 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:28.427644    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vg4t5" podStartSLOduration=16.632511678 podStartE2EDuration="18.427626373s" totalImagesPullingTime="1.795114695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:28.427182855 +0000 UTC m=+244.438421464" watchObservedRunningTime="2026-07-28 01:08:28.427626373 +0000 UTC m=+244.438864982"
Jul 28 01:08:30.327883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.327791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:30.328316 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.327919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:30.328576 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.328558    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/197c23c3-1540-404a-91ff-5a62e4fa333f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:30.330351 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.330321    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/197c23c3-1540-404a-91ff-5a62e4fa333f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cd6hz\" (UID: \"197c23c3-1540-404a-91ff-5a62e4fa333f\") " pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:30.451168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.451137    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7gpmd\""
Jul 28 01:08:30.459301 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.459281    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cd6hz"
Jul 28 01:08:30.585848 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:30.585817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cd6hz"]
Jul 28 01:08:30.588526 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:30.588501    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197c23c3_1540_404a_91ff_5a62e4fa333f.slice/crio-c1387d4421504abaa69872f66f077b48ebcff19d2ef1ef16702dd1f3ba1cb324 WatchSource:0}: Error finding container c1387d4421504abaa69872f66f077b48ebcff19d2ef1ef16702dd1f3ba1cb324: Status 404 returned error can't find the container with id c1387d4421504abaa69872f66f077b48ebcff19d2ef1ef16702dd1f3ba1cb324
Jul 28 01:08:31.421534 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:31.421451    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cd6hz" event={"ID":"197c23c3-1540-404a-91ff-5a62e4fa333f","Type":"ContainerStarted","Data":"c1387d4421504abaa69872f66f077b48ebcff19d2ef1ef16702dd1f3ba1cb324"}
Jul 28 01:08:36.377323 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:36.377279    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:08:36.380216 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:36.380174    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15ae3bf-ee89-4a3f-9ae1-565c670d2b31-metrics-certs\") pod \"network-metrics-daemon-fnd2j\" (UID: \"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31\") " pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:08:36.441360 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:36.441328    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cd6hz" event={"ID":"197c23c3-1540-404a-91ff-5a62e4fa333f","Type":"ContainerStarted","Data":"2d30ac4abc48497f43913efafbb1ad6cc307dc0a7c6a2d3fb96874ac4b033182"}
Jul 28 01:08:36.460509 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:36.460485    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qvhk6\""
Jul 28 01:08:36.467926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:36.467907    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fnd2j"
Jul 28 01:08:36.607548 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:36.607508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fnd2j"]
Jul 28 01:08:36.611114 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:36.611083    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15ae3bf_ee89_4a3f_9ae1_565c670d2b31.slice/crio-78ea2781b4b5394045bac6350084ba74926ad5a623497aa3526f683a613d7be4 WatchSource:0}: Error finding container 78ea2781b4b5394045bac6350084ba74926ad5a623497aa3526f683a613d7be4: Status 404 returned error can't find the container with id 78ea2781b4b5394045bac6350084ba74926ad5a623497aa3526f683a613d7be4
Jul 28 01:08:37.445930 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:37.445886    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fnd2j" event={"ID":"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31","Type":"ContainerStarted","Data":"78ea2781b4b5394045bac6350084ba74926ad5a623497aa3526f683a613d7be4"}
Jul 28 01:08:39.455461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.455371    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fnd2j" event={"ID":"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31","Type":"ContainerStarted","Data":"44478c24ff4d39810bfe2bbca407a12d50908d1bcd5d5fbc33055fca307251b5"}
Jul 28 01:08:39.455461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.455445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fnd2j" event={"ID":"b15ae3bf-ee89-4a3f-9ae1-565c670d2b31","Type":"ContainerStarted","Data":"d1b561a69daabf20a3e0c27c35310530528259d3c91582f524bf811feb6d87a0"}
Jul 28 01:08:39.457294 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.457254    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cd6hz" event={"ID":"197c23c3-1540-404a-91ff-5a62e4fa333f","Type":"ContainerStarted","Data":"85a5a433aebc53d6687fe9c4cdcd88aaffe77447e7483545092576daa7e0545b"}
Jul 28 01:08:39.471443 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.471397    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fnd2j" podStartSLOduration=251.851966145 podStartE2EDuration="4m14.471384198s" totalImagesPullingTime="2.619418053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:39.469716352 +0000 UTC m=+255.480954973" watchObservedRunningTime="2026-07-28 01:08:39.471384198 +0000 UTC m=+255.482622806"
Jul 28 01:08:39.745635 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.745558    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"]
Jul 28 01:08:39.748846 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.748819    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:39.751771 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.751746    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-plwp4\""
Jul 28 01:08:39.751883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.751746    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 01:08:39.759360 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.759339    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"]
Jul 28 01:08:39.767425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.767404    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55bddfb586-4qmrj"]
Jul 28 01:08:39.809868 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.809833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6qkll\" (UID: \"536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:39.910362 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.910322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6qkll\" (UID: \"536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:39.913438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:39.913404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-6qkll\" (UID: \"536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:40.059381 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:40.059295    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:40.224823 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:40.224794    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"]
Jul 28 01:08:40.229230 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:40.229200    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536fa5ca_bba3_4d3e_9c67_0ce5a05d0ac7.slice/crio-66e29d97d1f630c227d99ba4b185fb6204c9f4d56667dfada74d12cf2ee1893f WatchSource:0}: Error finding container 66e29d97d1f630c227d99ba4b185fb6204c9f4d56667dfada74d12cf2ee1893f: Status 404 returned error can't find the container with id 66e29d97d1f630c227d99ba4b185fb6204c9f4d56667dfada74d12cf2ee1893f
Jul 28 01:08:40.462658 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:40.462609    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll" event={"ID":"536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7","Type":"ContainerStarted","Data":"66e29d97d1f630c227d99ba4b185fb6204c9f4d56667dfada74d12cf2ee1893f"}
Jul 28 01:08:43.474126 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.474094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cd6hz" event={"ID":"197c23c3-1540-404a-91ff-5a62e4fa333f","Type":"ContainerStarted","Data":"fd0dd7992d4d44fe45f5c5d293a8920b1db729f67143ba5b0227a98340683bf4"}
Jul 28 01:08:43.475568 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.475544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll" event={"ID":"536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7","Type":"ContainerStarted","Data":"77c6f2b2af814d577a491ee88db5f84a225a07fb1cdb7faec9debba1430ffc09"}
Jul 28 01:08:43.475693 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.475586    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:43.480299 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.480274    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll"
Jul 28 01:08:43.492702 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.492657    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cd6hz" podStartSLOduration=17.200675585 podStartE2EDuration="29.492639594s" totalImagesPullingTime="12.291964009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 01:08:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:43.491688756 +0000 UTC m=+259.502927364" watchObservedRunningTime="2026-07-28 01:08:43.492639594 +0000 UTC m=+259.503878203"
Jul 28 01:08:43.506550 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.506503    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-6qkll" podStartSLOduration=1.713372731 podStartE2EDuration="4.506487089s" totalImagesPullingTime="2.793114358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:43.505106211 +0000 UTC m=+259.516344832" watchObservedRunningTime="2026-07-28 01:08:43.506487089 +0000 UTC m=+259.517725698"
Jul 28 01:08:43.860177 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.860148    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-r896j"]
Jul 28 01:08:43.863398 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.863381    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:43.865846 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.865820    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 28 01:08:43.865957 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.865928    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 01:08:43.866058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.866013    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 01:08:43.866111 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.866101    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 01:08:43.866352 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.866330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 28 01:08:43.867178 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.867162    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k6q72\""
Jul 28 01:08:43.871958 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.871935    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-r896j"]
Jul 28 01:08:43.941542 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.941500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgzhr\" (UniqueName: \"kubernetes.io/projected/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-kube-api-access-lgzhr\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:43.941542 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.941546    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:43.941795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.941635    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:43.941795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:43.941699    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.042873 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.042826    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.043047 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.042891    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.043047 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.042923    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.043047 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.042957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgzhr\" (UniqueName: \"kubernetes.io/projected/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-kube-api-access-lgzhr\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.043570 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.043545    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.045450 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.045432    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.045544 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.045527    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.050819 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.050798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgzhr\" (UniqueName: \"kubernetes.io/projected/3adb5bd3-f307-42ae-bd22-3ca0724ea19c-kube-api-access-lgzhr\") pod \"prometheus-operator-84d69f794d-r896j\" (UID: \"3adb5bd3-f307-42ae-bd22-3ca0724ea19c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.173330 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.173201    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j"
Jul 28 01:08:44.299305 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.299280    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-r896j"]
Jul 28 01:08:44.301739 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:44.301715    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3adb5bd3_f307_42ae_bd22_3ca0724ea19c.slice/crio-e81280fc912b813c38653519c3b1a413f4fb2a4db000f43897eb5d94f66b1be5 WatchSource:0}: Error finding container e81280fc912b813c38653519c3b1a413f4fb2a4db000f43897eb5d94f66b1be5: Status 404 returned error can't find the container with id e81280fc912b813c38653519c3b1a413f4fb2a4db000f43897eb5d94f66b1be5
Jul 28 01:08:44.480705 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:44.480625    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j" event={"ID":"3adb5bd3-f307-42ae-bd22-3ca0724ea19c","Type":"ContainerStarted","Data":"e81280fc912b813c38653519c3b1a413f4fb2a4db000f43897eb5d94f66b1be5"}
Jul 28 01:08:46.488845 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:46.488811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j" event={"ID":"3adb5bd3-f307-42ae-bd22-3ca0724ea19c","Type":"ContainerStarted","Data":"6d606d3ed2a50007fa248bb4fe0a5b48e8e28dbe96e3bdff3d11483e7526c6f6"}
Jul 28 01:08:46.489252 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:46.488862    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j" event={"ID":"3adb5bd3-f307-42ae-bd22-3ca0724ea19c","Type":"ContainerStarted","Data":"68aa7a8f2b36a3494517795a6b1b992f60e88da049b10401dd9ae5dbae9ad5b4"}
Jul 28 01:08:46.507020 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:46.506954    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-r896j" podStartSLOduration=1.9763071399999999 podStartE2EDuration="3.506937673s" totalImagesPullingTime="1.530630533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:46.505335483 +0000 UTC m=+262.516574094" watchObservedRunningTime="2026-07-28 01:08:46.506937673 +0000 UTC m=+262.518176316"
Jul 28 01:08:49.772771 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:49.772736    2566 patch_prober.go:28] interesting pod/image-registry-55bddfb586-4qmrj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 01:08:49.773173 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:49.772787    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 01:08:53.028125 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.028088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"]
Jul 28 01:08:53.031516 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.031499    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.034226 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.034046    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 28 01:08:53.034413 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.034395    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 28 01:08:53.034522 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.034501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sf8bz\""
Jul 28 01:08:53.034684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.034656    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 28 01:08:53.035008 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.034990    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nvvjp"]
Jul 28 01:08:53.038032 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.038014    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.040279 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.040258    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 01:08:53.040378 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.040277    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 01:08:53.040443 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.040413    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 01:08:53.040606 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.040590    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pq5bc\""
Jul 28 01:08:53.041651 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.041634    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"]
Jul 28 01:08:53.112425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e1a85-7436-492a-8776-938f82ff4257-metrics-client-ca\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-tls\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl5q8\" (UniqueName: \"kubernetes.io/projected/5b320f00-bbc1-454e-b303-e1113262789c-kube-api-access-vl5q8\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112558    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksxgd\" (UniqueName: \"kubernetes.io/projected/ae5e1a85-7436-492a-8776-938f82ff4257-kube-api-access-ksxgd\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b320f00-bbc1-454e-b303-e1113262789c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112597    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112613    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-root\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-wtmp\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.112909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112744    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b320f00-bbc1-454e-b303-e1113262789c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.112909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-accelerators-collector-config\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-textfile\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.112909 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.112815    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-sys\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.213730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.213730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213732    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksxgd\" (UniqueName: \"kubernetes.io/projected/ae5e1a85-7436-492a-8776-938f82ff4257-kube-api-access-ksxgd\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.213945 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:53.213825    2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 28 01:08:53.213945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213847    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b320f00-bbc1-454e-b303-e1113262789c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.213945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.213945 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:53.213893    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-tls podName:5b320f00-bbc1-454e-b303-e1113262789c nodeName:}" failed. No retries permitted until 2026-07-28 01:08:53.713878279 +0000 UTC m=+269.725116866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-rz6tq" (UID: "5b320f00-bbc1-454e-b303-e1113262789c") : secret "kube-state-metrics-tls" not found
Jul 28 01:08:53.213945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-root\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.213945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213935    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.213962    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-wtmp\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-root\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b320f00-bbc1-454e-b303-e1113262789c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214083    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-accelerators-collector-config\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-textfile\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214142    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-sys\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-wtmp\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e1a85-7436-492a-8776-938f82ff4257-metrics-client-ca\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214199    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-tls\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl5q8\" (UniqueName: \"kubernetes.io/projected/5b320f00-bbc1-454e-b303-e1113262789c-kube-api-access-vl5q8\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.214785 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214502    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-textfile\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214785 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:53.214570    2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 01:08:53.214785 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:08:53.214618    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-tls podName:ae5e1a85-7436-492a-8776-938f82ff4257 nodeName:}" failed. No retries permitted until 2026-07-28 01:08:53.714602098 +0000 UTC m=+269.725840686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-tls") pod "node-exporter-nvvjp" (UID: "ae5e1a85-7436-492a-8776-938f82ff4257") : secret "node-exporter-tls" not found
Jul 28 01:08:53.214785 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214669    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae5e1a85-7436-492a-8776-938f82ff4257-sys\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.214785 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b320f00-bbc1-454e-b303-e1113262789c-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.214954 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214862    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5b320f00-bbc1-454e-b303-e1113262789c-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.214954 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214905    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae5e1a85-7436-492a-8776-938f82ff4257-metrics-client-ca\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.215047 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.214995    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.215274 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.215249    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-accelerators-collector-config\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.216558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.216535    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.216646 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.216571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.227151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.227121    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksxgd\" (UniqueName: \"kubernetes.io/projected/ae5e1a85-7436-492a-8776-938f82ff4257-kube-api-access-ksxgd\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.230149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.230122    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl5q8\" (UniqueName: \"kubernetes.io/projected/5b320f00-bbc1-454e-b303-e1113262789c-kube-api-access-vl5q8\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.718532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.718472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.718724 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.718568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-tls\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.720990 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.720938    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ae5e1a85-7436-492a-8776-938f82ff4257-node-exporter-tls\") pod \"node-exporter-nvvjp\" (UID: \"ae5e1a85-7436-492a-8776-938f82ff4257\") " pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.721220 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.721197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5b320f00-bbc1-454e-b303-e1113262789c-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-rz6tq\" (UID: \"5b320f00-bbc1-454e-b303-e1113262789c\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.941800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.941759    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"
Jul 28 01:08:53.948121 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:53.948101    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nvvjp"
Jul 28 01:08:53.959516 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:53.959478    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5e1a85_7436_492a_8776_938f82ff4257.slice/crio-01f6e134cbf1019425fe504436ffe94029dbaff1145d099c4a243f19e0b8402f WatchSource:0}: Error finding container 01f6e134cbf1019425fe504436ffe94029dbaff1145d099c4a243f19e0b8402f: Status 404 returned error can't find the container with id 01f6e134cbf1019425fe504436ffe94029dbaff1145d099c4a243f19e0b8402f
Jul 28 01:08:54.086760 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.086719    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq"]
Jul 28 01:08:54.091068 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:54.091039    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b320f00_bbc1_454e_b303_e1113262789c.slice/crio-da3a38f67181bc31dfed9b8ea870195c6dc6498accf9c96539e03a69ed4cabc7 WatchSource:0}: Error finding container da3a38f67181bc31dfed9b8ea870195c6dc6498accf9c96539e03a69ed4cabc7: Status 404 returned error can't find the container with id da3a38f67181bc31dfed9b8ea870195c6dc6498accf9c96539e03a69ed4cabc7
Jul 28 01:08:54.518461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.518420    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq" event={"ID":"5b320f00-bbc1-454e-b303-e1113262789c","Type":"ContainerStarted","Data":"da3a38f67181bc31dfed9b8ea870195c6dc6498accf9c96539e03a69ed4cabc7"}
Jul 28 01:08:54.519822 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.519788    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nvvjp" event={"ID":"ae5e1a85-7436-492a-8776-938f82ff4257","Type":"ContainerStarted","Data":"01f6e134cbf1019425fe504436ffe94029dbaff1145d099c4a243f19e0b8402f"}
Jul 28 01:08:54.691481 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.691448    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-75bd47b75f-l9ts9"]
Jul 28 01:08:54.695104 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.695078    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.697606 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.697560    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 01:08:54.698686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698495    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dhb6l\""
Jul 28 01:08:54.698686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698498    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 01:08:54.698686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698606    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 01:08:54.698686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698498    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 01:08:54.698686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698627    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 01:08:54.699012 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698930    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 01:08:54.699012 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.698948    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 01:08:54.703352 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.703246    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 01:08:54.705160 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.705139    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bd47b75f-l9ts9"]
Jul 28 01:08:54.727916 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.727878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-trusted-ca-bundle\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.727916 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.727927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-config\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.728191 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.728026    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-oauth-config\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.728191 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.728059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-serving-cert\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.728191 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.728148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-service-ca\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.728402 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.728221    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-oauth-serving-cert\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.728402 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.728262    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8vq8\" (UniqueName: \"kubernetes.io/projected/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-kube-api-access-k8vq8\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.828959 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.828876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8vq8\" (UniqueName: \"kubernetes.io/projected/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-kube-api-access-k8vq8\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.828959 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.828938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-trusted-ca-bundle\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.829225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.828983    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-config\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.829225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.829025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-oauth-config\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.829225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.829048    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-serving-cert\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.829225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.829126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-service-ca\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.829225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.829167    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-oauth-serving-cert\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.830033 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.829952    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-oauth-serving-cert\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.830163 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.830122    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-config\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.830163 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.830137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-trusted-ca-bundle\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.830552 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.830525    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-service-ca\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.832313 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.832286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-oauth-config\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.833426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.833407    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-serving-cert\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:54.839029 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:54.839005    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8vq8\" (UniqueName: \"kubernetes.io/projected/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-kube-api-access-k8vq8\") pod \"console-75bd47b75f-l9ts9\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") " pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:55.009660 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:55.009622    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:08:55.166504 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:55.166470    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-75bd47b75f-l9ts9"]
Jul 28 01:08:55.388695 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:55.388589    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f8fb8f_3c16_4cc7_b779_3a31df8f3523.slice/crio-663d7455884078aa8deb452213baa2f4b87abc585a051d0c507953c4e40040b0 WatchSource:0}: Error finding container 663d7455884078aa8deb452213baa2f4b87abc585a051d0c507953c4e40040b0: Status 404 returned error can't find the container with id 663d7455884078aa8deb452213baa2f4b87abc585a051d0c507953c4e40040b0
Jul 28 01:08:55.525339 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:55.525303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq" event={"ID":"5b320f00-bbc1-454e-b303-e1113262789c","Type":"ContainerStarted","Data":"77d3d122404f89893199087c634939700b2612293d2f0e1b0f6eed4cebff4e3f"}
Jul 28 01:08:55.526462 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:55.526437    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bd47b75f-l9ts9" event={"ID":"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523","Type":"ContainerStarted","Data":"663d7455884078aa8deb452213baa2f4b87abc585a051d0c507953c4e40040b0"}
Jul 28 01:08:55.527843 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:55.527822    2566 generic.go:356] "Generic (PLEG): container finished" podID="ae5e1a85-7436-492a-8776-938f82ff4257" containerID="f53cdff2ed84f5a110bca357cc0775acc9bde67254d0abff3923d5db4f886063" exitCode=0
Jul 28 01:08:55.527915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:55.527876    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nvvjp" event={"ID":"ae5e1a85-7436-492a-8776-938f82ff4257","Type":"ContainerDied","Data":"f53cdff2ed84f5a110bca357cc0775acc9bde67254d0abff3923d5db4f886063"}
Jul 28 01:08:56.534736 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:56.534700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq" event={"ID":"5b320f00-bbc1-454e-b303-e1113262789c","Type":"ContainerStarted","Data":"3985e2cdd4cae9f6821f73829cb4b875ee4494717ee4c97a367dfaae2c44b789"}
Jul 28 01:08:56.535247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:56.534776    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq" event={"ID":"5b320f00-bbc1-454e-b303-e1113262789c","Type":"ContainerStarted","Data":"e399665e78bbbbed12c62b3358d186a1cbe3318ca4bade307133458d27f59e27"}
Jul 28 01:08:56.537245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:56.537181    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nvvjp" event={"ID":"ae5e1a85-7436-492a-8776-938f82ff4257","Type":"ContainerStarted","Data":"6355c1e7b90d60eadb8a59811b8760928c34ec2caefe6e1e5ad6e81cc161a0f5"}
Jul 28 01:08:56.537378 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:56.537271    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nvvjp" event={"ID":"ae5e1a85-7436-492a-8776-938f82ff4257","Type":"ContainerStarted","Data":"79612b1aa0c50f143c3ba94f58facad0a0ac42c0183334c29d28b490be465ff4"}
Jul 28 01:08:56.552110 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:56.552054    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-rz6tq" podStartSLOduration=2.210586022 podStartE2EDuration="3.552036831s" totalImagesPullingTime="1.341450809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:56.551037642 +0000 UTC m=+272.562276254" watchObservedRunningTime="2026-07-28 01:08:56.552036831 +0000 UTC m=+272.563275441"
Jul 28 01:08:57.800449 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.800391    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nvvjp" podStartSLOduration=3.714909198 podStartE2EDuration="4.800369592s" totalImagesPullingTime="1.000696104s" totalInitContainerRuntime="84.76429ms" isStatefulPod=true podCreationTimestamp="2026-07-28 01:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:56.567294031 +0000 UTC m=+272.578532641" watchObservedRunningTime="2026-07-28 01:08:57.800369592 +0000 UTC m=+273.811608201"
Jul 28 01:08:57.800919 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.800593    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"]
Jul 28 01:08:57.809415 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.809231    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:57.813306 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.811831    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"]
Jul 28 01:08:57.813497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.813350    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 01:08:57.813497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.813417    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-77nk8\""
Jul 28 01:08:57.861311 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.861222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdgzt\" (UniqueName: \"kubernetes.io/projected/fd70dd90-a4be-46c5-9ab3-1381a361d9c7-kube-api-access-bdgzt\") pod \"monitoring-plugin-68b7fc9574-62sc8\" (UID: \"fd70dd90-a4be-46c5-9ab3-1381a361d9c7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:57.861484 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.861430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd70dd90-a4be-46c5-9ab3-1381a361d9c7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-62sc8\" (UID: \"fd70dd90-a4be-46c5-9ab3-1381a361d9c7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:57.962192 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.962156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd70dd90-a4be-46c5-9ab3-1381a361d9c7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-62sc8\" (UID: \"fd70dd90-a4be-46c5-9ab3-1381a361d9c7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:57.962384 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.962223    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdgzt\" (UniqueName: \"kubernetes.io/projected/fd70dd90-a4be-46c5-9ab3-1381a361d9c7-kube-api-access-bdgzt\") pod \"monitoring-plugin-68b7fc9574-62sc8\" (UID: \"fd70dd90-a4be-46c5-9ab3-1381a361d9c7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:57.965127 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.965100    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fd70dd90-a4be-46c5-9ab3-1381a361d9c7-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-62sc8\" (UID: \"fd70dd90-a4be-46c5-9ab3-1381a361d9c7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:57.972540 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:57.972511    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdgzt\" (UniqueName: \"kubernetes.io/projected/fd70dd90-a4be-46c5-9ab3-1381a361d9c7-kube-api-access-bdgzt\") pod \"monitoring-plugin-68b7fc9574-62sc8\" (UID: \"fd70dd90-a4be-46c5-9ab3-1381a361d9c7\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:58.123621 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:58.123584    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:08:58.294245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:58.294220    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"]
Jul 28 01:08:58.296930 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:08:58.296902    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd70dd90_a4be_46c5_9ab3_1381a361d9c7.slice/crio-1ee6ab57c996ec304c85ea323bf4f3da877c9d2a4439ec2cceac82b4a9d2ed7a WatchSource:0}: Error finding container 1ee6ab57c996ec304c85ea323bf4f3da877c9d2a4439ec2cceac82b4a9d2ed7a: Status 404 returned error can't find the container with id 1ee6ab57c996ec304c85ea323bf4f3da877c9d2a4439ec2cceac82b4a9d2ed7a
Jul 28 01:08:58.549218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:58.549122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bd47b75f-l9ts9" event={"ID":"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523","Type":"ContainerStarted","Data":"2fceedc4c138f601971e88e6c155330e22196c9621a4ea3907f4fe0c5f64e83f"}
Jul 28 01:08:58.551319 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:58.551277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8" event={"ID":"fd70dd90-a4be-46c5-9ab3-1381a361d9c7","Type":"ContainerStarted","Data":"1ee6ab57c996ec304c85ea323bf4f3da877c9d2a4439ec2cceac82b4a9d2ed7a"}
Jul 28 01:08:58.569872 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:58.569793    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-75bd47b75f-l9ts9" podStartSLOduration=1.797182192 podStartE2EDuration="4.569768284s" totalImagesPullingTime="2.772586092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:08:58.567413193 +0000 UTC m=+274.578651911" watchObservedRunningTime="2026-07-28 01:08:58.569768284 +0000 UTC m=+274.581006893"
Jul 28 01:08:59.772400 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:59.772362    2566 patch_prober.go:28] interesting pod/image-registry-55bddfb586-4qmrj container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 01:08:59.772817 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:08:59.772437    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 01:09:00.562814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:00.562780    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8" event={"ID":"fd70dd90-a4be-46c5-9ab3-1381a361d9c7","Type":"ContainerStarted","Data":"0a055d71682890c1a9abdb4c85bcd8e11d2bb982628d2eade7dc8a7b9b9befa2"}
Jul 28 01:09:00.563008 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:00.562992    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:09:00.567397 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:00.567375    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8"
Jul 28 01:09:00.578507 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:00.578459    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-62sc8" podStartSLOduration=1.9210420209999999 podStartE2EDuration="3.578440002s" totalImagesPullingTime="1.657397981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:00.577348863 +0000 UTC m=+276.588587492" watchObservedRunningTime="2026-07-28 01:09:00.578440002 +0000 UTC m=+276.589678610"
Jul 28 01:09:01.537807 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.537765    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7466fcc8f7-jcv29"]
Jul 28 01:09:01.541344 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.541321    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.549717 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.549688    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7466fcc8f7-jcv29"]
Jul 28 01:09:01.598337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-service-ca\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.598337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zlr2\" (UniqueName: \"kubernetes.io/projected/b6bed859-c520-41c6-8d68-00536130fbc7-kube-api-access-7zlr2\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.598533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598373    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-oauth-serving-cert\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.598533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-trusted-ca-bundle\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.598533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598506    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-oauth-config\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.598655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-serving-cert\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.598655 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.598574    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-console-config\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.699864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.699808    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-trusted-ca-bundle\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.699866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-oauth-config\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.699908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-serving-cert\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.699931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-console-config\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-service-ca\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zlr2\" (UniqueName: \"kubernetes.io/projected/b6bed859-c520-41c6-8d68-00536130fbc7-kube-api-access-7zlr2\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700099    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-oauth-serving-cert\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700844 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700802    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-console-config\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.700844 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700813    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-oauth-serving-cert\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.701001 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700814    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-trusted-ca-bundle\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.701001 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.700923    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-service-ca\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.702650 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.702627    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-oauth-config\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.702752 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.702735    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-serving-cert\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.731353 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.731320    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zlr2\" (UniqueName: \"kubernetes.io/projected/b6bed859-c520-41c6-8d68-00536130fbc7-kube-api-access-7zlr2\") pod \"console-7466fcc8f7-jcv29\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") " pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.850746 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.850712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:01.983040 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:01.982938    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7466fcc8f7-jcv29"]
Jul 28 01:09:01.985138 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:09:01.985113    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6bed859_c520_41c6_8d68_00536130fbc7.slice/crio-01099ab034193e546faccfc04f6307dd5618b03414cf063ed2ee6da919e7e181 WatchSource:0}: Error finding container 01099ab034193e546faccfc04f6307dd5618b03414cf063ed2ee6da919e7e181: Status 404 returned error can't find the container with id 01099ab034193e546faccfc04f6307dd5618b03414cf063ed2ee6da919e7e181
Jul 28 01:09:02.571338 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:02.571303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7466fcc8f7-jcv29" event={"ID":"b6bed859-c520-41c6-8d68-00536130fbc7","Type":"ContainerStarted","Data":"8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14"}
Jul 28 01:09:02.571741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:02.571351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7466fcc8f7-jcv29" event={"ID":"b6bed859-c520-41c6-8d68-00536130fbc7","Type":"ContainerStarted","Data":"01099ab034193e546faccfc04f6307dd5618b03414cf063ed2ee6da919e7e181"}
Jul 28 01:09:02.589442 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:02.589391    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7466fcc8f7-jcv29" podStartSLOduration=1.589375965 podStartE2EDuration="1.589375965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:09:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:02.588290231 +0000 UTC m=+278.599528867" watchObservedRunningTime="2026-07-28 01:09:02.589375965 +0000 UTC m=+278.600614576"
Jul 28 01:09:04.792835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:04.792784    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerName="registry" containerID="cri-o://690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f" gracePeriod=30
Jul 28 01:09:05.010624 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.010590    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:09:05.010778 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.010646    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:09:05.011887 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.011866    2566 patch_prober.go:28] interesting pod/console-75bd47b75f-l9ts9 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.19:8443/health\": dial tcp 10.133.0.19:8443: connect: connection refused" start-of-body=
Jul 28 01:09:05.011934 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.011907    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-75bd47b75f-l9ts9" podUID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" containerName="console" probeResult="failure" output="Get \"https://10.133.0.19:8443/health\": dial tcp 10.133.0.19:8443: connect: connection refused"
Jul 28 01:09:05.075892 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.075864    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:09:05.128820 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.128788    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129055 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.128849    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-trusted-ca\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129055 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.128883    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-image-registry-private-configuration\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129055 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129006    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-certificates\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129055 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129045    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-installation-pull-secrets\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129073    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4nz7\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-kube-api-access-m4nz7\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129118    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-bound-sa-token\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129138    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-ca-trust-extracted\") pod \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\" (UID: \"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7\") "
Jul 28 01:09:05.129407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129366    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:05.129538 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129519    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-trusted-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.129742 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.129590    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:05.131690 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.131667    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-kube-api-access-m4nz7" (OuterVolumeSpecName: "kube-api-access-m4nz7") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "kube-api-access-m4nz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:05.131793 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.131706    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:05.131793 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.131735    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:05.131793 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.131754    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:05.132402 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.132380    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:05.139175 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.139153    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" (UID: "bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:09:05.229910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229878    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-tls\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.229910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229909    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-image-registry-private-configuration\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.229910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229919    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-registry-certificates\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.230162 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229929    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-installation-pull-secrets\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.230162 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229938    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4nz7\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-kube-api-access-m4nz7\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.230162 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229947    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-bound-sa-token\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.230162 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.229955    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7-ca-trust-extracted\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:05.587883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.587845    2566 generic.go:356] "Generic (PLEG): container finished" podID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerID="690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f" exitCode=0
Jul 28 01:09:05.587883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.587888    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" event={"ID":"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7","Type":"ContainerDied","Data":"690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f"}
Jul 28 01:09:05.587883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.587892    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj"
Jul 28 01:09:05.588147 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.587927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55bddfb586-4qmrj" event={"ID":"bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7","Type":"ContainerDied","Data":"e3d5319179850495ab36c217fd5ce1d1757ff2bd363582b04b6ada1e00183e50"}
Jul 28 01:09:05.588147 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.588000    2566 scope.go:117] "RemoveContainer" containerID="690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f"
Jul 28 01:09:05.596548 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.596530    2566 scope.go:117] "RemoveContainer" containerID="690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f"
Jul 28 01:09:05.596818 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:09:05.596799    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f\": container with ID starting with 690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f not found: ID does not exist" containerID="690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f"
Jul 28 01:09:05.596898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.596830    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f"} err="failed to get container status \"690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f\": rpc error: code = NotFound desc = could not find container \"690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f\": container with ID starting with 690f7e3c4111b24af1b87ffcae9cdf22f74a24471dc15a4dbf2da067a81c094f not found: ID does not exist"
Jul 28 01:09:05.607396 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.607371    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55bddfb586-4qmrj"]
Jul 28 01:09:05.610869 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:05.610846    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55bddfb586-4qmrj"]
Jul 28 01:09:06.172877 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:09:06.172830    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wvg8g" podUID="d13e3477-1101-406b-a2a6-4fcd0e206eec"
Jul 28 01:09:06.560096 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:06.560021    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" path="/var/lib/kubelet/pods/bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7/volumes"
Jul 28 01:09:06.596803 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:06.596775    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:09:07.507772 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.507734    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75bd47b75f-l9ts9"]
Jul 28 01:09:07.537164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.537133    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55868cb44-h9vp4"]
Jul 28 01:09:07.537438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.537423    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerName="registry"
Jul 28 01:09:07.537438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.537437    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerName="registry"
Jul 28 01:09:07.537606 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.537501    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbb7d30-ccc4-4e34-9bbc-e620e1a91ce7" containerName="registry"
Jul 28 01:09:07.547150 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.547124    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.550780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.550752    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55868cb44-h9vp4"]
Jul 28 01:09:07.648270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648222    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-service-ca\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.648468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-serving-cert\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.648468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648358    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-oauth-config\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.648468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-oauth-serving-cert\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.648468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648441    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-config\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.648650 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-trusted-ca-bundle\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.648650 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.648532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vvct\" (UniqueName: \"kubernetes.io/projected/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-kube-api-access-6vvct\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.749747 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vvct\" (UniqueName: \"kubernetes.io/projected/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-kube-api-access-6vvct\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.749915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-service-ca\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.749915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-serving-cert\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.749915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749837    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-oauth-config\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.749915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-oauth-serving-cert\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.750153 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-config\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.750153 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.749944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-trusted-ca-bundle\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.750632 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.750602    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-service-ca\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.750632 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.750616    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-oauth-serving-cert\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.750892 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.750868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-trusted-ca-bundle\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.750959 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.750888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-config\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.752586 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.752554    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-oauth-config\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.752910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.752891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-serving-cert\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.758075 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.758011    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vvct\" (UniqueName: \"kubernetes.io/projected/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-kube-api-access-6vvct\") pod \"console-55868cb44-h9vp4\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") " pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.858834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.858780    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:07.989835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:07.989799    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55868cb44-h9vp4"]
Jul 28 01:09:07.993061 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:09:07.993027    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0ff0e9_64ca_4e02_8506_2740c1fc029b.slice/crio-bc37ea0f2c97f049dbe34eb37c3cf5530d85d399f99797201f91ad105c4a8c53 WatchSource:0}: Error finding container bc37ea0f2c97f049dbe34eb37c3cf5530d85d399f99797201f91ad105c4a8c53: Status 404 returned error can't find the container with id bc37ea0f2c97f049dbe34eb37c3cf5530d85d399f99797201f91ad105c4a8c53
Jul 28 01:09:08.606883 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:08.606846    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55868cb44-h9vp4" event={"ID":"8a0ff0e9-64ca-4e02-8506-2740c1fc029b","Type":"ContainerStarted","Data":"2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1"}
Jul 28 01:09:08.607476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:08.606902    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55868cb44-h9vp4" event={"ID":"8a0ff0e9-64ca-4e02-8506-2740c1fc029b","Type":"ContainerStarted","Data":"bc37ea0f2c97f049dbe34eb37c3cf5530d85d399f99797201f91ad105c4a8c53"}
Jul 28 01:09:08.649880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:08.649831    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55868cb44-h9vp4" podStartSLOduration=1.6498163049999999 podStartE2EDuration="1.649816305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:08.648942035 +0000 UTC m=+284.660180660" watchObservedRunningTime="2026-07-28 01:09:08.649816305 +0000 UTC m=+284.661054914"
Jul 28 01:09:09.867242 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:09.867205    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:09:09.869670 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:09.869644    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d13e3477-1101-406b-a2a6-4fcd0e206eec-metrics-tls\") pod \"dns-default-wvg8g\" (UID: \"d13e3477-1101-406b-a2a6-4fcd0e206eec\") " pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:09:09.899380 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:09.899348    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9pqrg\""
Jul 28 01:09:09.907418 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:09.907397    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:09:10.054533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:10.054508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvg8g"]
Jul 28 01:09:10.056850 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:09:10.056821    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd13e3477_1101_406b_a2a6_4fcd0e206eec.slice/crio-d02688a8b3c598058298bba92078ca525c41eafd25c8bb7b10b7842b312414b0 WatchSource:0}: Error finding container d02688a8b3c598058298bba92078ca525c41eafd25c8bb7b10b7842b312414b0: Status 404 returned error can't find the container with id d02688a8b3c598058298bba92078ca525c41eafd25c8bb7b10b7842b312414b0
Jul 28 01:09:10.615871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:10.615818    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvg8g" event={"ID":"d13e3477-1101-406b-a2a6-4fcd0e206eec","Type":"ContainerStarted","Data":"d02688a8b3c598058298bba92078ca525c41eafd25c8bb7b10b7842b312414b0"}
Jul 28 01:09:11.851384 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:11.851335    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:11.851811 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:11.851433    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:11.852823 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:11.852797    2566 patch_prober.go:28] interesting pod/console-7466fcc8f7-jcv29 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused" start-of-body=
Jul 28 01:09:11.852940 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:11.852871    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7466fcc8f7-jcv29" podUID="b6bed859-c520-41c6-8d68-00536130fbc7" containerName="console" probeResult="failure" output="Get \"https://10.133.0.21:8443/health\": dial tcp 10.133.0.21:8443: connect: connection refused"
Jul 28 01:09:12.625917 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:12.625879    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvg8g" event={"ID":"d13e3477-1101-406b-a2a6-4fcd0e206eec","Type":"ContainerStarted","Data":"f9679611f651930c9489572f19d7adb3a6397491a71adcff34d226bd86d563c2"}
Jul 28 01:09:12.626092 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:12.625930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvg8g" event={"ID":"d13e3477-1101-406b-a2a6-4fcd0e206eec","Type":"ContainerStarted","Data":"381b18a796ab63d97099083ee3378dbd09f96929a63cc539c3b9369e0e3b2845"}
Jul 28 01:09:12.626092 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:12.626010    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:09:12.642818 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:12.642769    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wvg8g" podStartSLOduration=251.803749772 podStartE2EDuration="4m13.6427499s" totalImagesPullingTime="1.839000128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:09:12.642684836 +0000 UTC m=+288.653923445" watchObservedRunningTime="2026-07-28 01:09:12.6427499 +0000 UTC m=+288.653988503"
Jul 28 01:09:17.859501 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:17.859459    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:17.859963 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:17.859558    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:17.860896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:17.860874    2566 patch_prober.go:28] interesting pod/console-55868cb44-h9vp4 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused" start-of-body=
Jul 28 01:09:17.860997 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:17.860917    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-55868cb44-h9vp4" podUID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" containerName="console" probeResult="failure" output="Get \"https://10.133.0.22:8443/health\": dial tcp 10.133.0.22:8443: connect: connection refused"
Jul 28 01:09:21.855117 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:21.855082    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:21.858771 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:21.858748    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:22.632305 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:22.632275    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wvg8g"
Jul 28 01:09:24.400455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:24.400431    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 01:09:27.863791 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:27.863760    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:27.868483 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:27.867535    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:09:27.920902 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:27.920869    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7466fcc8f7-jcv29"]
Jul 28 01:09:32.532724 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.532657    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-75bd47b75f-l9ts9" podUID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" containerName="console" containerID="cri-o://2fceedc4c138f601971e88e6c155330e22196c9621a4ea3907f4fe0c5f64e83f" gracePeriod=15
Jul 28 01:09:32.702127 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.702088    2566 generic.go:356] "Generic (PLEG): container finished" podID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" containerID="2fceedc4c138f601971e88e6c155330e22196c9621a4ea3907f4fe0c5f64e83f" exitCode=0
Jul 28 01:09:32.702359 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.702336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bd47b75f-l9ts9" event={"ID":"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523","Type":"ContainerDied","Data":"2fceedc4c138f601971e88e6c155330e22196c9621a4ea3907f4fe0c5f64e83f"}
Jul 28 01:09:32.816304 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.816215    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:09:32.960863 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.960830    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-config\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.960870    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-serving-cert\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.960945    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-service-ca\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.960962    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-trusted-ca-bundle\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961000    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-oauth-serving-cert\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961021    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-oauth-config\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961044    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8vq8\" (UniqueName: \"kubernetes.io/projected/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-kube-api-access-k8vq8\") pod \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\" (UID: \"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523\") "
Jul 28 01:09:32.961372 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961338    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-config" (OuterVolumeSpecName: "console-config") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:32.961419 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961364    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-service-ca" (OuterVolumeSpecName: "service-ca") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:32.961483 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961421    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:32.961687 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.961660    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:32.963391 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.963366    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-kube-api-access-k8vq8" (OuterVolumeSpecName: "kube-api-access-k8vq8") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "kube-api-access-k8vq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:32.963743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.963726    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:32.963805 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:32.963752    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" (UID: "b5f8fb8f-3c16-4cc7-b779-3a31df8f3523"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:33.062360 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062323    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.062360 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062356    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.062360 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062372    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.062597 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062383    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.062597 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062395    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.062597 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062406    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.062597 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.062416    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8vq8\" (UniqueName: \"kubernetes.io/projected/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523-kube-api-access-k8vq8\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:33.707100 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.707067    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-75bd47b75f-l9ts9"
Jul 28 01:09:33.707100 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.707074    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-75bd47b75f-l9ts9" event={"ID":"b5f8fb8f-3c16-4cc7-b779-3a31df8f3523","Type":"ContainerDied","Data":"663d7455884078aa8deb452213baa2f4b87abc585a051d0c507953c4e40040b0"}
Jul 28 01:09:33.707584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.707137    2566 scope.go:117] "RemoveContainer" containerID="2fceedc4c138f601971e88e6c155330e22196c9621a4ea3907f4fe0c5f64e83f"
Jul 28 01:09:33.728787 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.728764    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-75bd47b75f-l9ts9"]
Jul 28 01:09:33.731995 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:33.731954    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-75bd47b75f-l9ts9"]
Jul 28 01:09:34.560604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:34.560574    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" path="/var/lib/kubelet/pods/b5f8fb8f-3c16-4cc7-b779-3a31df8f3523/volumes"
Jul 28 01:09:34.711668 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:34.711636    2566 generic.go:356] "Generic (PLEG): container finished" podID="253a805e-0ba8-4c8d-ab32-776f7eeb93ab" containerID="3bfcc03f35c8bac50cfdbab68c08c98c4b4c8531666c78a10cbb6153fd575841" exitCode=0
Jul 28 01:09:34.712115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:34.711708    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m68h5" event={"ID":"253a805e-0ba8-4c8d-ab32-776f7eeb93ab","Type":"ContainerDied","Data":"3bfcc03f35c8bac50cfdbab68c08c98c4b4c8531666c78a10cbb6153fd575841"}
Jul 28 01:09:34.712115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:34.712100    2566 scope.go:117] "RemoveContainer" containerID="3bfcc03f35c8bac50cfdbab68c08c98c4b4c8531666c78a10cbb6153fd575841"
Jul 28 01:09:35.521187 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:35.521155    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b598fd8c5-s7dmr_2e916bb7-6c9e-40e5-9ca7-a7d242bd9384/router/0.log"
Jul 28 01:09:35.629151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:35.629126    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zhqzt_c966ed32-b941-4d8a-a3aa-449f757957b6/serve-healthcheck-canary/0.log"
Jul 28 01:09:35.721656 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:35.721620    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-m68h5" event={"ID":"253a805e-0ba8-4c8d-ab32-776f7eeb93ab","Type":"ContainerStarted","Data":"d7101f3532d52ba6c2a87a20175c6c842b288f4e1e44b180ea59fb120d3f3988"}
Jul 28 01:09:35.831635 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:35.831556    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-vg4t5_7a320d79-4cb8-439d-8827-32318d0987a1/cluster-samples-operator/0.log"
Jul 28 01:09:36.029715 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:36.029681    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-vg4t5_7a320d79-4cb8-439d-8827-32318d0987a1/cluster-samples-operator-watch/0.log"
Jul 28 01:09:39.738457 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:39.738427    2566 generic.go:356] "Generic (PLEG): container finished" podID="106ccc7e-fbc5-4b90-8eee-829b1146ed20" containerID="81b6834af5cadf4751d18bc6c70dca4d7d6e457b33f24ee7fd5bf500564e60db" exitCode=0
Jul 28 01:09:39.738907 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:39.738505    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s" event={"ID":"106ccc7e-fbc5-4b90-8eee-829b1146ed20","Type":"ContainerDied","Data":"81b6834af5cadf4751d18bc6c70dca4d7d6e457b33f24ee7fd5bf500564e60db"}
Jul 28 01:09:39.738907 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:39.738827    2566 scope.go:117] "RemoveContainer" containerID="81b6834af5cadf4751d18bc6c70dca4d7d6e457b33f24ee7fd5bf500564e60db"
Jul 28 01:09:40.743679 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:40.743646    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-6764s" event={"ID":"106ccc7e-fbc5-4b90-8eee-829b1146ed20","Type":"ContainerStarted","Data":"bdc5a04c7999bc664fd5c92e1f8c3b57b26c2264f5dd869de926fd0747d26eb1"}
Jul 28 01:09:52.940588 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:52.940541    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7466fcc8f7-jcv29" podUID="b6bed859-c520-41c6-8d68-00536130fbc7" containerName="console" containerID="cri-o://8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14" gracePeriod=15
Jul 28 01:09:53.190367 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.190342    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:53.226681 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226416    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-oauth-serving-cert\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.226681 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226475    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-trusted-ca-bundle\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.226681 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226519    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-console-config\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.226681 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226552    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-serving-cert\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.226681 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226591    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zlr2\" (UniqueName: \"kubernetes.io/projected/b6bed859-c520-41c6-8d68-00536130fbc7-kube-api-access-7zlr2\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.226681 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226654    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-oauth-config\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.227197 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.226701    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-service-ca\") pod \"b6bed859-c520-41c6-8d68-00536130fbc7\" (UID: \"b6bed859-c520-41c6-8d68-00536130fbc7\") "
Jul 28 01:09:53.227345 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.227302    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-service-ca" (OuterVolumeSpecName: "service-ca") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:53.228133 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.227741    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:53.228133 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.227870    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:53.228686 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.228478    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-console-config" (OuterVolumeSpecName: "console-config") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:09:53.230115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.230078    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:53.230226 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.230165    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:09:53.230954 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.230931    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6bed859-c520-41c6-8d68-00536130fbc7-kube-api-access-7zlr2" (OuterVolumeSpecName: "kube-api-access-7zlr2") pod "b6bed859-c520-41c6-8d68-00536130fbc7" (UID: "b6bed859-c520-41c6-8d68-00536130fbc7"). InnerVolumeSpecName "kube-api-access-7zlr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:09:53.327617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327578    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.327617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327610    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.327617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327620    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.327617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327631    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6bed859-c520-41c6-8d68-00536130fbc7-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.327885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327639    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.327885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327647    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zlr2\" (UniqueName: \"kubernetes.io/projected/b6bed859-c520-41c6-8d68-00536130fbc7-kube-api-access-7zlr2\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.327885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.327656    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6bed859-c520-41c6-8d68-00536130fbc7-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:09:53.796951 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.796917    2566 generic.go:356] "Generic (PLEG): container finished" podID="b6bed859-c520-41c6-8d68-00536130fbc7" containerID="8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14" exitCode=0
Jul 28 01:09:53.797151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.796991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7466fcc8f7-jcv29" event={"ID":"b6bed859-c520-41c6-8d68-00536130fbc7","Type":"ContainerDied","Data":"8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14"}
Jul 28 01:09:53.797151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.797011    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7466fcc8f7-jcv29"
Jul 28 01:09:53.797151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.797034    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7466fcc8f7-jcv29" event={"ID":"b6bed859-c520-41c6-8d68-00536130fbc7","Type":"ContainerDied","Data":"01099ab034193e546faccfc04f6307dd5618b03414cf063ed2ee6da919e7e181"}
Jul 28 01:09:53.797151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.797054    2566 scope.go:117] "RemoveContainer" containerID="8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14"
Jul 28 01:09:53.805872 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.805855    2566 scope.go:117] "RemoveContainer" containerID="8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14"
Jul 28 01:09:53.806209 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:09:53.806188    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14\": container with ID starting with 8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14 not found: ID does not exist" containerID="8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14"
Jul 28 01:09:53.806268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.806219    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14"} err="failed to get container status \"8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14\": rpc error: code = NotFound desc = could not find container \"8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14\": container with ID starting with 8bc2b5c9c506d457d1647eccd4a45790d84324ec378aeffd60b74904657cff14 not found: ID does not exist"
Jul 28 01:09:53.817411 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.817366    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7466fcc8f7-jcv29"]
Jul 28 01:09:53.821763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:53.821740    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7466fcc8f7-jcv29"]
Jul 28 01:09:54.559666 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:09:54.559637    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6bed859-c520-41c6-8d68-00536130fbc7" path="/var/lib/kubelet/pods/b6bed859-c520-41c6-8d68-00536130fbc7/volumes"
Jul 28 01:10:18.108222 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108180    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65855d9b4d-l59cr"]
Jul 28 01:10:18.108885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108617    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6bed859-c520-41c6-8d68-00536130fbc7" containerName="console"
Jul 28 01:10:18.108885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108635    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6bed859-c520-41c6-8d68-00536130fbc7" containerName="console"
Jul 28 01:10:18.108885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108652    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" containerName="console"
Jul 28 01:10:18.108885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108660    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" containerName="console"
Jul 28 01:10:18.108885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108754    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f8fb8f-3c16-4cc7-b779-3a31df8f3523" containerName="console"
Jul 28 01:10:18.108885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.108766    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6bed859-c520-41c6-8d68-00536130fbc7" containerName="console"
Jul 28 01:10:18.111592 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.111570    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.121793 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.121767    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65855d9b4d-l59cr"]
Jul 28 01:10:18.224591 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224522    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-oauth-serving-cert\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.224774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224608    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-trusted-ca-bundle\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.224774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224635    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-console-config\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.224774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq6qv\" (UniqueName: \"kubernetes.io/projected/e21744ef-416b-4258-b310-84e3fc04f257-kube-api-access-zq6qv\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.224774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224711    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-serving-cert\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.224774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224732    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-oauth-config\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.224932 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.224778    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-service-ca\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326070 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6qv\" (UniqueName: \"kubernetes.io/projected/e21744ef-416b-4258-b310-84e3fc04f257-kube-api-access-zq6qv\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326081    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-serving-cert\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-oauth-config\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326155    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-service-ca\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326185    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-oauth-serving-cert\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326223    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-trusted-ca-bundle\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326246    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-console-config\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.326955 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.326930    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-oauth-serving-cert\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.327183 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.327158    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-service-ca\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.327324 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.327304    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-trusted-ca-bundle\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.327380 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.327311    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-console-config\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.328825 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.328798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-oauth-config\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.328908 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.328827    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-serving-cert\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.333890 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.333865    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6qv\" (UniqueName: \"kubernetes.io/projected/e21744ef-416b-4258-b310-84e3fc04f257-kube-api-access-zq6qv\") pod \"console-65855d9b4d-l59cr\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") " pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.421722 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.421607    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:18.550177 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.550144    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65855d9b4d-l59cr"]
Jul 28 01:10:18.553065 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:10:18.553040    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21744ef_416b_4258_b310_84e3fc04f257.slice/crio-888149801b64a66d8519bda422de09ec2786c8bd82bf5817cbd7bcfc1c4101e4 WatchSource:0}: Error finding container 888149801b64a66d8519bda422de09ec2786c8bd82bf5817cbd7bcfc1c4101e4: Status 404 returned error can't find the container with id 888149801b64a66d8519bda422de09ec2786c8bd82bf5817cbd7bcfc1c4101e4
Jul 28 01:10:18.891420 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.891384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65855d9b4d-l59cr" event={"ID":"e21744ef-416b-4258-b310-84e3fc04f257","Type":"ContainerStarted","Data":"a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11"}
Jul 28 01:10:18.891630 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.891436    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65855d9b4d-l59cr" event={"ID":"e21744ef-416b-4258-b310-84e3fc04f257","Type":"ContainerStarted","Data":"888149801b64a66d8519bda422de09ec2786c8bd82bf5817cbd7bcfc1c4101e4"}
Jul 28 01:10:18.908724 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:18.908671    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65855d9b4d-l59cr" podStartSLOduration=0.908655445 podStartE2EDuration="908.655445ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:10:18.906871046 +0000 UTC m=+354.918109668" watchObservedRunningTime="2026-07-28 01:10:18.908655445 +0000 UTC m=+354.919894054"
Jul 28 01:10:28.421904 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:28.421866    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:28.422344 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:28.421941    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:28.426598 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:28.426573    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:28.931838 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:28.931810    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:10:28.972776 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:28.972747    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55868cb44-h9vp4"]
Jul 28 01:10:53.995399 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:53.995330    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55868cb44-h9vp4" podUID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" containerName="console" containerID="cri-o://2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1" gracePeriod=15
Jul 28 01:10:54.236982 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.236951    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:10:54.328962 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.328863    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-serving-cert\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.328962 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.328931    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-service-ca\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.329293 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329026    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-oauth-serving-cert\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.329293 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329048    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-config\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.329293 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329075    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vvct\" (UniqueName: \"kubernetes.io/projected/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-kube-api-access-6vvct\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.329293 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329105    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-oauth-config\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.329293 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329134    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-trusted-ca-bundle\") pod \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\" (UID: \"8a0ff0e9-64ca-4e02-8506-2740c1fc029b\") "
Jul 28 01:10:54.329532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329410    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-service-ca" (OuterVolumeSpecName: "service-ca") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:10:54.329532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329420    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:10:54.329532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329464    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-config" (OuterVolumeSpecName: "console-config") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:10:54.329723 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.329702    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:10:54.331301 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.331271    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:10:54.331301 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.331290    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:10:54.331530 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.331511    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-kube-api-access-6vvct" (OuterVolumeSpecName: "kube-api-access-6vvct") pod "8a0ff0e9-64ca-4e02-8506-2740c1fc029b" (UID: "8a0ff0e9-64ca-4e02-8506-2740c1fc029b"). InnerVolumeSpecName "kube-api-access-6vvct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:10:54.429947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.429906    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:54.429947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.429940    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:54.429947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.429955    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vvct\" (UniqueName: \"kubernetes.io/projected/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-kube-api-access-6vvct\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:54.430254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.429997    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:54.430254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.430009    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:54.430254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.430021    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:54.430254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:54.430031    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8a0ff0e9-64ca-4e02-8506-2740c1fc029b-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:10:55.026069 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.026032    2566 generic.go:356] "Generic (PLEG): container finished" podID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" containerID="2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1" exitCode=0
Jul 28 01:10:55.026565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.026086    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55868cb44-h9vp4"
Jul 28 01:10:55.026565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.026127    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55868cb44-h9vp4" event={"ID":"8a0ff0e9-64ca-4e02-8506-2740c1fc029b","Type":"ContainerDied","Data":"2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1"}
Jul 28 01:10:55.026565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.026191    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55868cb44-h9vp4" event={"ID":"8a0ff0e9-64ca-4e02-8506-2740c1fc029b","Type":"ContainerDied","Data":"bc37ea0f2c97f049dbe34eb37c3cf5530d85d399f99797201f91ad105c4a8c53"}
Jul 28 01:10:55.026565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.026222    2566 scope.go:117] "RemoveContainer" containerID="2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1"
Jul 28 01:10:55.036578 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.036558    2566 scope.go:117] "RemoveContainer" containerID="2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1"
Jul 28 01:10:55.037049 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:10:55.037020    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1\": container with ID starting with 2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1 not found: ID does not exist" containerID="2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1"
Jul 28 01:10:55.037400 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.037058    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1"} err="failed to get container status \"2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1\": rpc error: code = NotFound desc = could not find container \"2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1\": container with ID starting with 2a2fe10780d9331b8c27e5662a9db97147af482d8e489c614870e9ab9d6962a1 not found: ID does not exist"
Jul 28 01:10:55.040721 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.040700    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55868cb44-h9vp4"]
Jul 28 01:10:55.042889 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:55.042868    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55868cb44-h9vp4"]
Jul 28 01:10:56.560190 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:10:56.560155    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" path="/var/lib/kubelet/pods/8a0ff0e9-64ca-4e02-8506-2740c1fc029b/volumes"
Jul 28 01:11:23.706428 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.706392    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"]
Jul 28 01:11:23.706843 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.706675    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" containerName="console"
Jul 28 01:11:23.706843 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.706688    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" containerName="console"
Jul 28 01:11:23.706843 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.706748    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a0ff0e9-64ca-4e02-8506-2740c1fc029b" containerName="console"
Jul 28 01:11:23.709646 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.709628    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:23.712432 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.712402    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-g8kxk\""
Jul 28 01:11:23.712984 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.712948    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 01:11:23.713180 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.713168    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:11:23.740257 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.740225    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"]
Jul 28 01:11:23.761917 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.761884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d21bdeb7-42db-4b1d-ac0f-8da175c08f9e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lvgrp\" (UID: \"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:23.762092 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.761947    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sktkl\" (UniqueName: \"kubernetes.io/projected/d21bdeb7-42db-4b1d-ac0f-8da175c08f9e-kube-api-access-sktkl\") pod \"cert-manager-operator-controller-manager-55c444bf86-lvgrp\" (UID: \"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:23.862523 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.862490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d21bdeb7-42db-4b1d-ac0f-8da175c08f9e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lvgrp\" (UID: \"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:23.862697 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.862534    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sktkl\" (UniqueName: \"kubernetes.io/projected/d21bdeb7-42db-4b1d-ac0f-8da175c08f9e-kube-api-access-sktkl\") pod \"cert-manager-operator-controller-manager-55c444bf86-lvgrp\" (UID: \"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:23.862850 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.862832    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d21bdeb7-42db-4b1d-ac0f-8da175c08f9e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lvgrp\" (UID: \"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:23.873080 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:23.873055    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sktkl\" (UniqueName: \"kubernetes.io/projected/d21bdeb7-42db-4b1d-ac0f-8da175c08f9e-kube-api-access-sktkl\") pod \"cert-manager-operator-controller-manager-55c444bf86-lvgrp\" (UID: \"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:24.018842 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:24.018750    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"
Jul 28 01:11:24.157313 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:24.157266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp"]
Jul 28 01:11:24.161840 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:11:24.161816    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21bdeb7_42db_4b1d_ac0f_8da175c08f9e.slice/crio-51d6da45c655234c38e980293faced3360fa87a061025949884d892d96acb590 WatchSource:0}: Error finding container 51d6da45c655234c38e980293faced3360fa87a061025949884d892d96acb590: Status 404 returned error can't find the container with id 51d6da45c655234c38e980293faced3360fa87a061025949884d892d96acb590
Jul 28 01:11:24.164378 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:24.164361    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:11:25.140749 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:25.140703    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp" event={"ID":"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e","Type":"ContainerStarted","Data":"51d6da45c655234c38e980293faced3360fa87a061025949884d892d96acb590"}
Jul 28 01:11:27.152019 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.151963    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp" event={"ID":"d21bdeb7-42db-4b1d-ac0f-8da175c08f9e","Type":"ContainerStarted","Data":"b10e724c784bcb0b77d92afda31c168dcbbb6a8f93cea4f83296eb30ff20a3c1"}
Jul 28 01:11:27.170709 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.170643    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lvgrp" podStartSLOduration=1.294304125 podStartE2EDuration="4.170604061s" totalImagesPullingTime="2.876299936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:11:27.170245712 +0000 UTC m=+423.181484334" watchObservedRunningTime="2026-07-28 01:11:27.170604061 +0000 UTC m=+423.181842670"
Jul 28 01:11:27.738628 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.738590    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"]
Jul 28 01:11:27.742097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.742078    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.748756 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.748733    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-skk9x\""
Jul 28 01:11:27.749170 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.749150    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 28 01:11:27.749248 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.749157    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 28 01:11:27.759025 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.758996    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"]
Jul 28 01:11:27.794928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.794891    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.795147 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.795100    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkg68\" (UniqueName: \"kubernetes.io/projected/748b3d23-67df-42d1-85b5-b627aeb64c2b-kube-api-access-tkg68\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.795264 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.795245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.895814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.895769    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.895814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.895820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.896096 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.895870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkg68\" (UniqueName: \"kubernetes.io/projected/748b3d23-67df-42d1-85b5-b627aeb64c2b-kube-api-access-tkg68\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.896188 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.896169    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.896246 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.896199    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:27.904492 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:27.904430    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkg68\" (UniqueName: \"kubernetes.io/projected/748b3d23-67df-42d1-85b5-b627aeb64c2b-kube-api-access-tkg68\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:28.051148 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:28.051058    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:28.188851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:28.188797    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"]
Jul 28 01:11:28.191229 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:11:28.191196    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748b3d23_67df_42d1_85b5_b627aeb64c2b.slice/crio-00700791e247338d22e957c8f2cefb0b711b0476d52933a2f310e164e628ac91 WatchSource:0}: Error finding container 00700791e247338d22e957c8f2cefb0b711b0476d52933a2f310e164e628ac91: Status 404 returned error can't find the container with id 00700791e247338d22e957c8f2cefb0b711b0476d52933a2f310e164e628ac91
Jul 28 01:11:29.164029 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.163994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v" event={"ID":"748b3d23-67df-42d1-85b5-b627aeb64c2b","Type":"ContainerStarted","Data":"00700791e247338d22e957c8f2cefb0b711b0476d52933a2f310e164e628ac91"}
Jul 28 01:11:29.769583 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.769543    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"]
Jul 28 01:11:29.772955 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.772935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:29.775420 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.775374    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 28 01:11:29.775536 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.775518    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-82hzh\""
Jul 28 01:11:29.775602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.775518    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 28 01:11:29.782442 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.782108    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"]
Jul 28 01:11:29.915609 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.915570    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8bcfa94-498a-473b-b0fa-b461c1dd9f45-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dlsqk\" (UID: \"e8bcfa94-498a-473b-b0fa-b461c1dd9f45\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:29.915806 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:29.915620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkv7x\" (UniqueName: \"kubernetes.io/projected/e8bcfa94-498a-473b-b0fa-b461c1dd9f45-kube-api-access-pkv7x\") pod \"cert-manager-cainjector-559cdd8c87-dlsqk\" (UID: \"e8bcfa94-498a-473b-b0fa-b461c1dd9f45\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:30.016385 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:30.016348    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8bcfa94-498a-473b-b0fa-b461c1dd9f45-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dlsqk\" (UID: \"e8bcfa94-498a-473b-b0fa-b461c1dd9f45\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:30.016692 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:30.016399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkv7x\" (UniqueName: \"kubernetes.io/projected/e8bcfa94-498a-473b-b0fa-b461c1dd9f45-kube-api-access-pkv7x\") pod \"cert-manager-cainjector-559cdd8c87-dlsqk\" (UID: \"e8bcfa94-498a-473b-b0fa-b461c1dd9f45\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:30.024563 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:30.024490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e8bcfa94-498a-473b-b0fa-b461c1dd9f45-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dlsqk\" (UID: \"e8bcfa94-498a-473b-b0fa-b461c1dd9f45\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:30.024712 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:30.024669    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkv7x\" (UniqueName: \"kubernetes.io/projected/e8bcfa94-498a-473b-b0fa-b461c1dd9f45-kube-api-access-pkv7x\") pod \"cert-manager-cainjector-559cdd8c87-dlsqk\" (UID: \"e8bcfa94-498a-473b-b0fa-b461c1dd9f45\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:30.098135 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:30.098091    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"
Jul 28 01:11:30.253280 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:30.253138    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk"]
Jul 28 01:11:30.256240 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:11:30.256208    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8bcfa94_498a_473b_b0fa_b461c1dd9f45.slice/crio-71a0c80a12643e3895c8dfc2527f49fd4a634d4128ea9fc97d7c425bec58ad66 WatchSource:0}: Error finding container 71a0c80a12643e3895c8dfc2527f49fd4a634d4128ea9fc97d7c425bec58ad66: Status 404 returned error can't find the container with id 71a0c80a12643e3895c8dfc2527f49fd4a634d4128ea9fc97d7c425bec58ad66
Jul 28 01:11:31.178748 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:31.178709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk" event={"ID":"e8bcfa94-498a-473b-b0fa-b461c1dd9f45","Type":"ContainerStarted","Data":"71a0c80a12643e3895c8dfc2527f49fd4a634d4128ea9fc97d7c425bec58ad66"}
Jul 28 01:11:35.199837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:35.199799    2566 generic.go:356] "Generic (PLEG): container finished" podID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerID="58bc9da64946c65334069a2484ed02f3e642fcc7e0cfa49513bca94d9dce664a" exitCode=0
Jul 28 01:11:35.200419 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:35.199893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v" event={"ID":"748b3d23-67df-42d1-85b5-b627aeb64c2b","Type":"ContainerDied","Data":"58bc9da64946c65334069a2484ed02f3e642fcc7e0cfa49513bca94d9dce664a"}
Jul 28 01:11:35.201828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:35.201803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk" event={"ID":"e8bcfa94-498a-473b-b0fa-b461c1dd9f45","Type":"ContainerStarted","Data":"6665e35ead0669905e2c30bd415392323a71f39e7b61de69df9de0cbad446449"}
Jul 28 01:11:35.235324 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:35.235217    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dlsqk" podStartSLOduration=1.513240905 podStartE2EDuration="6.235195723s" totalImagesPullingTime="4.721954818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:11:35.234196901 +0000 UTC m=+431.245435511" watchObservedRunningTime="2026-07-28 01:11:35.235195723 +0000 UTC m=+431.246434331"
Jul 28 01:11:41.229065 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:41.229023    2566 generic.go:356] "Generic (PLEG): container finished" podID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerID="189a438b7dbf3670e66004ddb5dc04918715367a333007ca226f49afe1abfa5c" exitCode=0
Jul 28 01:11:41.229561 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:41.229094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v" event={"ID":"748b3d23-67df-42d1-85b5-b627aeb64c2b","Type":"ContainerDied","Data":"189a438b7dbf3670e66004ddb5dc04918715367a333007ca226f49afe1abfa5c"}
Jul 28 01:11:49.266098 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:49.266064    2566 generic.go:356] "Generic (PLEG): container finished" podID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerID="cc8c4c87adfde03224c3b9ada458273278a5b46abeee738ee8dc16189f173a25" exitCode=0
Jul 28 01:11:49.266524 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:49.266167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v" event={"ID":"748b3d23-67df-42d1-85b5-b627aeb64c2b","Type":"ContainerDied","Data":"cc8c4c87adfde03224c3b9ada458273278a5b46abeee738ee8dc16189f173a25"}
Jul 28 01:11:50.408016 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.407961    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:50.596795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.596762    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkg68\" (UniqueName: \"kubernetes.io/projected/748b3d23-67df-42d1-85b5-b627aeb64c2b-kube-api-access-tkg68\") pod \"748b3d23-67df-42d1-85b5-b627aeb64c2b\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") "
Jul 28 01:11:50.597016 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.596806    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-util\") pod \"748b3d23-67df-42d1-85b5-b627aeb64c2b\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") "
Jul 28 01:11:50.597016 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.596824    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-bundle\") pod \"748b3d23-67df-42d1-85b5-b627aeb64c2b\" (UID: \"748b3d23-67df-42d1-85b5-b627aeb64c2b\") "
Jul 28 01:11:50.597246 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.597220    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-bundle" (OuterVolumeSpecName: "bundle") pod "748b3d23-67df-42d1-85b5-b627aeb64c2b" (UID: "748b3d23-67df-42d1-85b5-b627aeb64c2b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:11:50.599194 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.599167    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/748b3d23-67df-42d1-85b5-b627aeb64c2b-kube-api-access-tkg68" (OuterVolumeSpecName: "kube-api-access-tkg68") pod "748b3d23-67df-42d1-85b5-b627aeb64c2b" (UID: "748b3d23-67df-42d1-85b5-b627aeb64c2b"). InnerVolumeSpecName "kube-api-access-tkg68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:11:50.601412 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.601388    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-util" (OuterVolumeSpecName: "util") pod "748b3d23-67df-42d1-85b5-b627aeb64c2b" (UID: "748b3d23-67df-42d1-85b5-b627aeb64c2b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:11:50.697758 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.697715    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkg68\" (UniqueName: \"kubernetes.io/projected/748b3d23-67df-42d1-85b5-b627aeb64c2b-kube-api-access-tkg68\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:11:50.697758 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.697752    2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-util\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:11:50.697758 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:50.697766    2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/748b3d23-67df-42d1-85b5-b627aeb64c2b-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:11:51.276798 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:51.276758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v" event={"ID":"748b3d23-67df-42d1-85b5-b627aeb64c2b","Type":"ContainerDied","Data":"00700791e247338d22e957c8f2cefb0b711b0476d52933a2f310e164e628ac91"}
Jul 28 01:11:51.276798 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:51.276789    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fw624v"
Jul 28 01:11:51.276798 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:51.276806    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="00700791e247338d22e957c8f2cefb0b711b0476d52933a2f310e164e628ac91"
Jul 28 01:11:56.565061 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565035    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"]
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565339    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="util"
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565351    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="util"
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565360    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="extract"
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565366    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="extract"
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565379    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="pull"
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565385    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="pull"
Jul 28 01:11:56.565455 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.565441    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="748b3d23-67df-42d1-85b5-b627aeb64c2b" containerName="extract"
Jul 28 01:11:56.567299 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.567283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.569632 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.569602    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hjdss\""
Jul 28 01:11:56.570618 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.570601    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 01:11:56.570741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.570642    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 01:11:56.575710 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.575684    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"]
Jul 28 01:11:56.639043 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.639004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5597c68-6e5f-48f4-9b4e-836b9aef3bcc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8kvrn\" (UID: \"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.639230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.639051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8r8c\" (UniqueName: \"kubernetes.io/projected/f5597c68-6e5f-48f4-9b4e-836b9aef3bcc-kube-api-access-q8r8c\") pod \"openshift-lws-operator-fd8ccff4c-8kvrn\" (UID: \"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.740140 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.740108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5597c68-6e5f-48f4-9b4e-836b9aef3bcc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8kvrn\" (UID: \"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.740140 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.740147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8r8c\" (UniqueName: \"kubernetes.io/projected/f5597c68-6e5f-48f4-9b4e-836b9aef3bcc-kube-api-access-q8r8c\") pod \"openshift-lws-operator-fd8ccff4c-8kvrn\" (UID: \"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.740490 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.740470    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f5597c68-6e5f-48f4-9b4e-836b9aef3bcc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8kvrn\" (UID: \"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.747825 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.747801    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8r8c\" (UniqueName: \"kubernetes.io/projected/f5597c68-6e5f-48f4-9b4e-836b9aef3bcc-kube-api-access-q8r8c\") pod \"openshift-lws-operator-fd8ccff4c-8kvrn\" (UID: \"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:56.876637 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:56.876599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"
Jul 28 01:11:57.003308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:57.003275    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn"]
Jul 28 01:11:57.006956 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:11:57.006933    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5597c68_6e5f_48f4_9b4e_836b9aef3bcc.slice/crio-76655852a1912338890ea36b7ce83dd6a2d09471bb0907b179b37e6fa03d3370 WatchSource:0}: Error finding container 76655852a1912338890ea36b7ce83dd6a2d09471bb0907b179b37e6fa03d3370: Status 404 returned error can't find the container with id 76655852a1912338890ea36b7ce83dd6a2d09471bb0907b179b37e6fa03d3370
Jul 28 01:11:57.302910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:57.302826    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn" event={"ID":"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc","Type":"ContainerStarted","Data":"76655852a1912338890ea36b7ce83dd6a2d09471bb0907b179b37e6fa03d3370"}
Jul 28 01:11:59.313538 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:59.313504    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn" event={"ID":"f5597c68-6e5f-48f4-9b4e-836b9aef3bcc","Type":"ContainerStarted","Data":"cdc872ae5c0b835246d739258dec9aae7b35f6d83b921cdda1eac9891857b070"}
Jul 28 01:11:59.328283 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:11:59.328231    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8kvrn" podStartSLOduration=1.755363038 podStartE2EDuration="3.328210802s" totalImagesPullingTime="1.572847764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:11:59.327924373 +0000 UTC m=+455.339162981" watchObservedRunningTime="2026-07-28 01:11:59.328210802 +0000 UTC m=+455.339449412"
Jul 28 01:12:12.350168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.350126    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"]
Jul 28 01:12:12.352459 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.352440    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.356053 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.356027    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 01:12:12.356210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.356117    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 01:12:12.356210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.356124    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-v7bh5\""
Jul 28 01:12:12.356210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.356156    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 01:12:12.364867 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.364838    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"]
Jul 28 01:12:12.462345 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.462307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/506f057a-5674-4078-a16f-1022149fb34e-metrics-cert\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.462571 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.462355    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wpcv\" (UniqueName: \"kubernetes.io/projected/506f057a-5674-4078-a16f-1022149fb34e-kube-api-access-9wpcv\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.462571 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.462466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/506f057a-5674-4078-a16f-1022149fb34e-manager-config\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.462571 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.462526    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/506f057a-5674-4078-a16f-1022149fb34e-cert\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.563906 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.563870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/506f057a-5674-4078-a16f-1022149fb34e-manager-config\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.564149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.563916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/506f057a-5674-4078-a16f-1022149fb34e-cert\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.564149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.563948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/506f057a-5674-4078-a16f-1022149fb34e-metrics-cert\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.564149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.563988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wpcv\" (UniqueName: \"kubernetes.io/projected/506f057a-5674-4078-a16f-1022149fb34e-kube-api-access-9wpcv\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.564688 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.564658    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/506f057a-5674-4078-a16f-1022149fb34e-manager-config\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.566605 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.566582    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/506f057a-5674-4078-a16f-1022149fb34e-cert\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.566692 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.566669    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/506f057a-5674-4078-a16f-1022149fb34e-metrics-cert\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.574219 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.574195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wpcv\" (UniqueName: \"kubernetes.io/projected/506f057a-5674-4078-a16f-1022149fb34e-kube-api-access-9wpcv\") pod \"lws-controller-manager-6d99f9899c-mbvlv\" (UID: \"506f057a-5674-4078-a16f-1022149fb34e\") " pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.666824 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.666746    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:12.802733 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:12.802697    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"]
Jul 28 01:12:12.805471 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:12:12.805445    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod506f057a_5674_4078_a16f_1022149fb34e.slice/crio-497ccc2abb13bc37793c40dc2661e412b83a837565507de7d44f8dabe308e6f8 WatchSource:0}: Error finding container 497ccc2abb13bc37793c40dc2661e412b83a837565507de7d44f8dabe308e6f8: Status 404 returned error can't find the container with id 497ccc2abb13bc37793c40dc2661e412b83a837565507de7d44f8dabe308e6f8
Jul 28 01:12:13.375079 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:13.375045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv" event={"ID":"506f057a-5674-4078-a16f-1022149fb34e","Type":"ContainerStarted","Data":"497ccc2abb13bc37793c40dc2661e412b83a837565507de7d44f8dabe308e6f8"}
Jul 28 01:12:14.381279 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:14.381247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv" event={"ID":"506f057a-5674-4078-a16f-1022149fb34e","Type":"ContainerStarted","Data":"c49d54e14e5ca855a4c7b5f583f8a58a08aa925cb05f43f95f69b26ec993ca5f"}
Jul 28 01:12:14.381671 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:14.381384    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:14.397564 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:14.397469    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv" podStartSLOduration=0.896294128 podStartE2EDuration="2.397446875s" totalImagesPullingTime="1.501152747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:12:14.397267561 +0000 UTC m=+470.408506196" watchObservedRunningTime="2026-07-28 01:12:14.397446875 +0000 UTC m=+470.408685485"
Jul 28 01:12:16.462933 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.462896    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"]
Jul 28 01:12:16.465080 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.465058    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.467586 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.467565    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 01:12:16.467704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.467563    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 01:12:16.467842 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.467826    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-b7f4g\""
Jul 28 01:12:16.467910 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.467828    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 01:12:16.468072 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.468058    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 01:12:16.486438 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.486404    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"]
Jul 28 01:12:16.497782 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.497752    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx59b\" (UniqueName: \"kubernetes.io/projected/c7832476-bdc4-48b4-887c-8215a257964a-kube-api-access-bx59b\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.497948 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.497791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7832476-bdc4-48b4-887c-8215a257964a-tmp\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.497948 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.497814    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7832476-bdc4-48b4-887c-8215a257964a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.497948 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.497925    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7832476-bdc4-48b4-887c-8215a257964a-manifests\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.498077 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.498033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7832476-bdc4-48b4-887c-8215a257964a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.598987 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.598933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7832476-bdc4-48b4-887c-8215a257964a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.599182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.599000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx59b\" (UniqueName: \"kubernetes.io/projected/c7832476-bdc4-48b4-887c-8215a257964a-kube-api-access-bx59b\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.599182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.599027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7832476-bdc4-48b4-887c-8215a257964a-tmp\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.599182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.599052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7832476-bdc4-48b4-887c-8215a257964a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.599182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.599097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7832476-bdc4-48b4-887c-8215a257964a-manifests\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.599632 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.599593    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c7832476-bdc4-48b4-887c-8215a257964a-manifests\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.601844 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.601813    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7832476-bdc4-48b4-887c-8215a257964a-tmp\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.602013 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.601901    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c7832476-bdc4-48b4-887c-8215a257964a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.602013 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.601935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7832476-bdc4-48b4-887c-8215a257964a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.609495 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.609451    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx59b\" (UniqueName: \"kubernetes.io/projected/c7832476-bdc4-48b4-887c-8215a257964a-kube-api-access-bx59b\") pod \"opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv\" (UID: \"c7832476-bdc4-48b4-887c-8215a257964a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.775950 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.775859    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:16.922801 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:16.918460    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"]
Jul 28 01:12:17.397184 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:17.397141    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv" event={"ID":"c7832476-bdc4-48b4-887c-8215a257964a","Type":"ContainerStarted","Data":"b0cf5ac12dd564b54943fbf1b1c1bb139706eb63b2f501a8cc0af2225a6ee048"}
Jul 28 01:12:21.417365 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:21.417331    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv" event={"ID":"c7832476-bdc4-48b4-887c-8215a257964a","Type":"ContainerStarted","Data":"739f11d8773537a70b13c14bad9c2fadb8651c9bd960bc281408dcc71a28ce91"}
Jul 28 01:12:22.422872 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:22.422830    2566 generic.go:356] "Generic (PLEG): container finished" podID="c7832476-bdc4-48b4-887c-8215a257964a" containerID="739f11d8773537a70b13c14bad9c2fadb8651c9bd960bc281408dcc71a28ce91" exitCode=0
Jul 28 01:12:22.423282 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:22.422926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv" event={"ID":"c7832476-bdc4-48b4-887c-8215a257964a","Type":"ContainerDied","Data":"739f11d8773537a70b13c14bad9c2fadb8651c9bd960bc281408dcc71a28ce91"}
Jul 28 01:12:23.429546 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:23.429501    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv" event={"ID":"c7832476-bdc4-48b4-887c-8215a257964a","Type":"ContainerStarted","Data":"73331cfddec9605f760a127c459399e8d3d30c795bd70f6203c46e3968107a9e"}
Jul 28 01:12:23.430014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:23.429635    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:23.450650 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:23.450145    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv" podStartSLOduration=2.468546526 podStartE2EDuration="7.450131284s" totalImagesPullingTime="3.914759321s" totalInitContainerRuntime="1.066825437s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:12:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:12:23.448369907 +0000 UTC m=+479.459608517" watchObservedRunningTime="2026-07-28 01:12:23.450131284 +0000 UTC m=+479.461369893"
Jul 28 01:12:25.387581 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:25.387550    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d99f9899c-mbvlv"
Jul 28 01:12:34.435163 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:34.435132    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv"
Jul 28 01:12:53.984741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.984703    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"]
Jul 28 01:12:53.992257 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.992232    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:53.995051 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.995026    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 01:12:53.995206 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.995071    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 01:12:53.995431 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.995416    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 01:12:53.996365 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.996347    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m4x25\""
Jul 28 01:12:53.996469 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:53.996380    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 01:12:54.002231 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.002192    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"]
Jul 28 01:12:54.121257 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.121456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.121456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121302    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpv9q\" (UniqueName: \"kubernetes.io/projected/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-kube-api-access-cpv9q\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.121456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.121456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121393    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.121456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121441    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.121682 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.121492    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.222780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.222780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222783    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.223101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222826    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.223101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222848    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpv9q\" (UniqueName: \"kubernetes.io/projected/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-kube-api-access-cpv9q\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.223101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.223101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.223101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.222992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.224042 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.224014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.225473 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.225445    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.225643 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.225624    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.225713 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.225628    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.225713 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.225667    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.234016 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.233958    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpv9q\" (UniqueName: \"kubernetes.io/projected/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-kube-api-access-cpv9q\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.234238 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.234219    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c6fc6dc-d5b9-413e-b2d3-a4d17848f224-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-8p648\" (UID: \"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.303155 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.303071    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:54.463305 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.463280    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"]
Jul 28 01:12:54.464917 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:12:54.464880    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6fc6dc_d5b9_413e_b2d3_a4d17848f224.slice/crio-d76f4243ce8523145e915e206b8a98527e480c786f0029079f3420773ce77916 WatchSource:0}: Error finding container d76f4243ce8523145e915e206b8a98527e480c786f0029079f3420773ce77916: Status 404 returned error can't find the container with id d76f4243ce8523145e915e206b8a98527e480c786f0029079f3420773ce77916
Jul 28 01:12:54.568625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:54.568535    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648" event={"ID":"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224","Type":"ContainerStarted","Data":"d76f4243ce8523145e915e206b8a98527e480c786f0029079f3420773ce77916"}
Jul 28 01:12:57.263447 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:57.263383    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 01:12:57.263832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:57.263485    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 28 01:12:57.605499 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:57.605463    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648" event={"ID":"1c6fc6dc-d5b9-413e-b2d3-a4d17848f224","Type":"ContainerStarted","Data":"78c8c04159ed509dea3b5960e05ee0731ca085ba67f2b9d60054ad1a0dc67286"}
Jul 28 01:12:57.605719 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:57.605524    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:12:57.627392 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:57.627336    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648" podStartSLOduration=1.8313115500000001 podStartE2EDuration="4.627317908s" totalImagesPullingTime="2.796006358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:12:57.626339314 +0000 UTC m=+513.637577927" watchObservedRunningTime="2026-07-28 01:12:57.627317908 +0000 UTC m=+513.638556510"
Jul 28 01:12:58.613307 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:12:58.613254    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-8p648"
Jul 28 01:14:27.840190 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:27.840147    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65855d9b4d-l59cr"]
Jul 28 01:14:52.861684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:52.861609    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65855d9b4d-l59cr" podUID="e21744ef-416b-4258-b310-84e3fc04f257" containerName="console" containerID="cri-o://a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11" gracePeriod=15
Jul 28 01:14:53.099601 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.099573    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:14:53.122007 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.121899    2566 generic.go:356] "Generic (PLEG): container finished" podID="e21744ef-416b-4258-b310-84e3fc04f257" containerID="a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11" exitCode=0
Jul 28 01:14:53.122007 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.121940    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65855d9b4d-l59cr"
Jul 28 01:14:53.122233 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.122010    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65855d9b4d-l59cr" event={"ID":"e21744ef-416b-4258-b310-84e3fc04f257","Type":"ContainerDied","Data":"a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11"}
Jul 28 01:14:53.122233 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.122068    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65855d9b4d-l59cr" event={"ID":"e21744ef-416b-4258-b310-84e3fc04f257","Type":"ContainerDied","Data":"888149801b64a66d8519bda422de09ec2786c8bd82bf5817cbd7bcfc1c4101e4"}
Jul 28 01:14:53.122233 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.122094    2566 scope.go:117] "RemoveContainer" containerID="a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11"
Jul 28 01:14:53.124584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124414    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-trusted-ca-bundle\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124457    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-service-ca\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124498    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zq6qv\" (UniqueName: \"kubernetes.io/projected/e21744ef-416b-4258-b310-84e3fc04f257-kube-api-access-zq6qv\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124559    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-serving-cert\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124830 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124590    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-oauth-serving-cert\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124830 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124638    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-console-config\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124830 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124660    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-oauth-config\") pod \"e21744ef-416b-4258-b310-84e3fc04f257\" (UID: \"e21744ef-416b-4258-b310-84e3fc04f257\") "
Jul 28 01:14:53.124959 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124829    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:14:53.124959 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.124880    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-service-ca" (OuterVolumeSpecName: "service-ca") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:14:53.125556 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.125501    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-console-config" (OuterVolumeSpecName: "console-config") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:14:53.125856 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.125818    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:14:53.127555 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.127519    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:14:53.128139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.128110    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21744ef-416b-4258-b310-84e3fc04f257-kube-api-access-zq6qv" (OuterVolumeSpecName: "kube-api-access-zq6qv") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "kube-api-access-zq6qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:14:53.128265 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.128189    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e21744ef-416b-4258-b310-84e3fc04f257" (UID: "e21744ef-416b-4258-b310-84e3fc04f257"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:14:53.132679 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.132656    2566 scope.go:117] "RemoveContainer" containerID="a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11"
Jul 28 01:14:53.133091 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:14:53.132949    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11\": container with ID starting with a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11 not found: ID does not exist" containerID="a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11"
Jul 28 01:14:53.133091 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.132993    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11"} err="failed to get container status \"a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11\": rpc error: code = NotFound desc = could not find container \"a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11\": container with ID starting with a3a392f070137fb5e7443d968d6253468cafd7ceb5dc25229bda86b5788d9b11 not found: ID does not exist"
Jul 28 01:14:53.225567 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225531    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.225567 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225559    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-oauth-serving-cert\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.225567 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225568    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-console-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.225567 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225577    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e21744ef-416b-4258-b310-84e3fc04f257-console-oauth-config\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.225823 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225585    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-trusted-ca-bundle\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.225823 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225593    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e21744ef-416b-4258-b310-84e3fc04f257-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.225823 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.225600    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zq6qv\" (UniqueName: \"kubernetes.io/projected/e21744ef-416b-4258-b310-84e3fc04f257-kube-api-access-zq6qv\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:14:53.451819 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.451785    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65855d9b4d-l59cr"]
Jul 28 01:14:53.457945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:53.457925    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65855d9b4d-l59cr"]
Jul 28 01:14:54.559777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:14:54.559746    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e21744ef-416b-4258-b310-84e3fc04f257" path="/var/lib/kubelet/pods/e21744ef-416b-4258-b310-84e3fc04f257/volumes"
Jul 28 01:15:44.475157 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.475122    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:15:44.475559 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.475422    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e21744ef-416b-4258-b310-84e3fc04f257" containerName="console"
Jul 28 01:15:44.475559 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.475433    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e21744ef-416b-4258-b310-84e3fc04f257" containerName="console"
Jul 28 01:15:44.475559 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.475497    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e21744ef-416b-4258-b310-84e3fc04f257" containerName="console"
Jul 28 01:15:44.479767 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.479750    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.482205 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.482183    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 01:15:44.482338 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.482212    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 01:15:44.482338 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.482259    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 01:15:44.483172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.483158    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8k9rr\""
Jul 28 01:15:44.485726 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.485700    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:15:44.512625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.512591    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:15:44.656939 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.656900    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4bhq\" (UniqueName: \"kubernetes.io/projected/d6934dbe-c0be-48e6-918e-b35819dd9b61-kube-api-access-p4bhq\") pod \"limitador-limitador-54d6d988d5-cpd2s\" (UID: \"d6934dbe-c0be-48e6-918e-b35819dd9b61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.657139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.656954    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d6934dbe-c0be-48e6-918e-b35819dd9b61-config-file\") pod \"limitador-limitador-54d6d988d5-cpd2s\" (UID: \"d6934dbe-c0be-48e6-918e-b35819dd9b61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.758493 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.758391    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4bhq\" (UniqueName: \"kubernetes.io/projected/d6934dbe-c0be-48e6-918e-b35819dd9b61-kube-api-access-p4bhq\") pod \"limitador-limitador-54d6d988d5-cpd2s\" (UID: \"d6934dbe-c0be-48e6-918e-b35819dd9b61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.758493 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.758440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d6934dbe-c0be-48e6-918e-b35819dd9b61-config-file\") pod \"limitador-limitador-54d6d988d5-cpd2s\" (UID: \"d6934dbe-c0be-48e6-918e-b35819dd9b61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.759142 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.759118    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d6934dbe-c0be-48e6-918e-b35819dd9b61-config-file\") pod \"limitador-limitador-54d6d988d5-cpd2s\" (UID: \"d6934dbe-c0be-48e6-918e-b35819dd9b61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.769425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.769404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4bhq\" (UniqueName: \"kubernetes.io/projected/d6934dbe-c0be-48e6-918e-b35819dd9b61-kube-api-access-p4bhq\") pod \"limitador-limitador-54d6d988d5-cpd2s\" (UID: \"d6934dbe-c0be-48e6-918e-b35819dd9b61\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.791291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.791257    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:44.921981 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:44.921943    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:15:44.924474 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:15:44.924446    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6934dbe_c0be_48e6_918e_b35819dd9b61.slice/crio-0b4513adb6608a9ffe27ecc63377bfe6559ba4d336eec219f2d70be5aca7fca5 WatchSource:0}: Error finding container 0b4513adb6608a9ffe27ecc63377bfe6559ba4d336eec219f2d70be5aca7fca5: Status 404 returned error can't find the container with id 0b4513adb6608a9ffe27ecc63377bfe6559ba4d336eec219f2d70be5aca7fca5
Jul 28 01:15:45.354678 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:45.354641    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s" event={"ID":"d6934dbe-c0be-48e6-918e-b35819dd9b61","Type":"ContainerStarted","Data":"0b4513adb6608a9ffe27ecc63377bfe6559ba4d336eec219f2d70be5aca7fca5"}
Jul 28 01:15:48.371396 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:48.371356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s" event={"ID":"d6934dbe-c0be-48e6-918e-b35819dd9b61","Type":"ContainerStarted","Data":"d9205da805eaa2610d77a1cbb63631fb2bb5b079fe6b3dd178f75a9c7f271b4e"}
Jul 28 01:15:48.371830 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:48.371470    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:15:48.389011 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:48.388937    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s" podStartSLOduration=1.8656745780000001 podStartE2EDuration="4.38892034s" totalImagesPullingTime="2.523245762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:15:48.386822211 +0000 UTC m=+684.398060832" watchObservedRunningTime="2026-07-28 01:15:48.38892034 +0000 UTC m=+684.400158981"
Jul 28 01:15:59.377026 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:15:59.376994    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"
Jul 28 01:17:00.690143 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:00.690104    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:17:29.736926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:29.736892    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:17:37.320487 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.320444    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"]
Jul 28 01:17:37.323936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.323911    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.327600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.327576    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 01:17:37.327600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.327592    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 01:17:37.327834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.327646    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bb82g\""
Jul 28 01:17:37.327834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.327591    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 01:17:37.336779 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.336747    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"]
Jul 28 01:17:37.401149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.401105    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.401331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.401156    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.401331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.401238    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.401331 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.401285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krlpv\" (UniqueName: \"kubernetes.io/projected/a060e5f2-30a7-4fcc-8475-624bf962a250-kube-api-access-krlpv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.401432 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.401344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a060e5f2-30a7-4fcc-8475-624bf962a250-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.401432 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.401380    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502459 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502459 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502464    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502666    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krlpv\" (UniqueName: \"kubernetes.io/projected/a060e5f2-30a7-4fcc-8475-624bf962a250-kube-api-access-krlpv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502746    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a060e5f2-30a7-4fcc-8475-624bf962a250-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502814    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502881 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502855    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.502933 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.502898    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.503137 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.503117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.505072 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.505049    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a060e5f2-30a7-4fcc-8475-624bf962a250-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.505254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.505235    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a060e5f2-30a7-4fcc-8475-624bf962a250-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.511424 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.511398    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krlpv\" (UniqueName: \"kubernetes.io/projected/a060e5f2-30a7-4fcc-8475-624bf962a250-kube-api-access-krlpv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mptxp\" (UID: \"a060e5f2-30a7-4fcc-8475-624bf962a250\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.636243 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.636200    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:37.777428 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.777398    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"]
Jul 28 01:17:37.779444 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:17:37.779417    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda060e5f2_30a7_4fcc_8475_624bf962a250.slice/crio-4959d3e0ce3540f73c668a58fc748817525fdbc5c65473955a4dd3b81476b948 WatchSource:0}: Error finding container 4959d3e0ce3540f73c668a58fc748817525fdbc5c65473955a4dd3b81476b948: Status 404 returned error can't find the container with id 4959d3e0ce3540f73c668a58fc748817525fdbc5c65473955a4dd3b81476b948
Jul 28 01:17:37.780806 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.780790    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:17:37.866815 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:37.866776    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp" event={"ID":"a060e5f2-30a7-4fcc-8475-624bf962a250","Type":"ContainerStarted","Data":"4959d3e0ce3540f73c668a58fc748817525fdbc5c65473955a4dd3b81476b948"}
Jul 28 01:17:38.035176 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:38.035088    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:17:39.883717 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:39.883675    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp" event={"ID":"a060e5f2-30a7-4fcc-8475-624bf962a250","Type":"ContainerStarted","Data":"d4b1b21829a4d6d515dc1346d707c3347570ddc7df60a0ae3911c1dab79e123f"}
Jul 28 01:17:39.902783 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:39.902729    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp" podStartSLOduration=1.852523264 podStartE2EDuration="2.902711777s" totalImagesPullingTime="1.050188513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:17:39.900893769 +0000 UTC m=+795.912132380" watchObservedRunningTime="2026-07-28 01:17:39.902711777 +0000 UTC m=+795.913950386"
Jul 28 01:17:47.636815 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:47.636732    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:47.637304 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:47.636825    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:47.648855 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:47.648832    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:47.931779 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:47.931699    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mptxp"
Jul 28 01:17:48.537178 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:48.537143    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:17:58.442796 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:17:58.442752    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:18:10.625769 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:18:10.625726    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:18:21.932313 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:18:21.932272    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:19:30.184778 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:19:30.184739    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:19:39.872203 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:19:39.872165    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:19:48.893640 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:19:48.893601    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:21:31.441397 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:21:31.441365    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:21:40.337664 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:21:40.337626    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:22:09.423101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:22:09.423066    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:22:19.133190 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:22:19.133156    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:22:27.435647 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:22:27.435611    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:22:36.632609 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:22:36.632572    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:22:46.327004 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:22:46.326954    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:22:56.145506 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:22:56.145464    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:23:07.021165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:23:07.021128    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:23:17.044028 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:23:17.043990    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:23:38.438499 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:23:38.438466    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:23:47.435848 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:23:47.435763    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:24:13.733748 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:24:13.733704    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:24:23.830415 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:24:23.830374    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:24:33.133466 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:24:33.133426    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:24:51.734168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:24:51.734130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:25:01.445637 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:25:01.445600    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:25:10.691054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:25:10.691018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:25:20.130169 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:25:20.130042    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:25:29.527196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:25:29.527157    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:27:16.825525 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:27:16.825487    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:27:27.836957 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:27:27.836904    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:27:40.839596 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:27:40.839564    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:27:50.733928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:27:50.733894    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:28:01.335828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:28:01.335785    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:28:12.436998 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:28:12.436946    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:28:22.640240 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:28:22.640204    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:28:32.032671 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:28:32.032621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:28:42.540444 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:28:42.540400    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:28:53.642834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:28:53.642795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:29:03.930369 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:03.930327    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:29:24.736311 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:24.736276    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:29:34.335320 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:34.335281    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:29:46.334250 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.334169    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"]
Jul 28 01:29:46.338021 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.338001    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.340537 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.340517    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-0e04e8ff-kserve-self-signed-certs\""
Jul 28 01:29:46.357337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.357312    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"]
Jul 28 01:29:46.405832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.405796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-model-cache\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.406028 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.405837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgshp\" (UniqueName: \"kubernetes.io/projected/71fe1053-b099-41ff-a33d-14948f90ca37-kube-api-access-jgshp\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.406028 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.405865    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-dshm\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.406028 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.405996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71fe1053-b099-41ff-a33d-14948f90ca37-tls-certs\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.406028 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.406026    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-home\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.406188 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.406078    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-tmp-dir\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.506953 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.506915    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-model-cache\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507155 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.506997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgshp\" (UniqueName: \"kubernetes.io/projected/71fe1053-b099-41ff-a33d-14948f90ca37-kube-api-access-jgshp\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507155 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507032    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-dshm\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507155 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507098    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71fe1053-b099-41ff-a33d-14948f90ca37-tls-certs\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507380 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507345    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-model-cache\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507380 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507358    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-home\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507577 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-tmp-dir\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507713 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507690    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-home\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.507951 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.507736    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-tmp-dir\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.509566 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.509546    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-dshm\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.509718 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.509702    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71fe1053-b099-41ff-a33d-14948f90ca37-tls-certs\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.514466 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.514445    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgshp\" (UniqueName: \"kubernetes.io/projected/71fe1053-b099-41ff-a33d-14948f90ca37-kube-api-access-jgshp\") pod \"e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") " pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.647094 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.647059    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:46.779784 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:46.779759    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"]
Jul 28 01:29:46.782555 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:29:46.782526    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71fe1053_b099_41ff_a33d_14948f90ca37.slice/crio-c501a2796bafbec451f953d80d5ea831eb2a67cfda6bdf471eb66d33e4224455 WatchSource:0}: Error finding container c501a2796bafbec451f953d80d5ea831eb2a67cfda6bdf471eb66d33e4224455: Status 404 returned error can't find the container with id c501a2796bafbec451f953d80d5ea831eb2a67cfda6bdf471eb66d33e4224455
Jul 28 01:29:47.108370 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:47.108336    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2" event={"ID":"71fe1053-b099-41ff-a33d-14948f90ca37","Type":"ContainerStarted","Data":"4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122"}
Jul 28 01:29:47.108370 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:47.108377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2" event={"ID":"71fe1053-b099-41ff-a33d-14948f90ca37","Type":"ContainerStarted","Data":"c501a2796bafbec451f953d80d5ea831eb2a67cfda6bdf471eb66d33e4224455"}
Jul 28 01:29:47.124898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:47.124846    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2" podStartSLOduration=1.124823965 podStartE2EDuration="1.124823965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:47.123212387 +0000 UTC m=+1523.134451000" watchObservedRunningTime="2026-07-28 01:29:47.124823965 +0000 UTC m=+1523.136062574"
Jul 28 01:29:49.432072 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:49.432038    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:29:53.423320 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.423279    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"]
Jul 28 01:29:53.426852 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.426831    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.429231 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.429210    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-0e04e8ff-kserve-self-signed-certs\""
Jul 28 01:29:53.436554 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.436531    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"]
Jul 28 01:29:53.571880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.571850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-tmp-dir\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.572097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.571890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-home\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.572097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.571915    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-model-cache\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.572097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.571955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkmf8\" (UniqueName: \"kubernetes.io/projected/d01afab1-4667-45f7-8029-1dc927c3270e-kube-api-access-mkmf8\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.572097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.571995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-dshm\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.572097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.572017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01afab1-4667-45f7-8029-1dc927c3270e-tls-certs\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673419    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-home\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-model-cache\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkmf8\" (UniqueName: \"kubernetes.io/projected/d01afab1-4667-45f7-8029-1dc927c3270e-kube-api-access-mkmf8\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673529    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-dshm\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673890 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01afab1-4667-45f7-8029-1dc927c3270e-tls-certs\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673890 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-tmp-dir\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.673890 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673878    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-home\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.674082 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.673949    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-model-cache\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.674281 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.674262    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-tmp-dir\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.675981 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.675947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-dshm\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.676309 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.676287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01afab1-4667-45f7-8029-1dc927c3270e-tls-certs\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.681227 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.681200    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkmf8\" (UniqueName: \"kubernetes.io/projected/d01afab1-4667-45f7-8029-1dc927c3270e-kube-api-access-mkmf8\") pod \"e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") " pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.736896 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.736861    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:29:53.873349 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:53.873323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"]
Jul 28 01:29:53.875922 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:29:53.875889    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd01afab1_4667_45f7_8029_1dc927c3270e.slice/crio-49fe46d87c6ac23b877081b7f4939c6258df8f0801624e792c71928d903e7023 WatchSource:0}: Error finding container 49fe46d87c6ac23b877081b7f4939c6258df8f0801624e792c71928d903e7023: Status 404 returned error can't find the container with id 49fe46d87c6ac23b877081b7f4939c6258df8f0801624e792c71928d903e7023
Jul 28 01:29:54.141988 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:54.141932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d" event={"ID":"d01afab1-4667-45f7-8029-1dc927c3270e","Type":"ContainerStarted","Data":"4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db"}
Jul 28 01:29:54.142165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:54.141994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d" event={"ID":"d01afab1-4667-45f7-8029-1dc927c3270e","Type":"ContainerStarted","Data":"49fe46d87c6ac23b877081b7f4939c6258df8f0801624e792c71928d903e7023"}
Jul 28 01:29:54.160153 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:54.160111    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d" podStartSLOduration=1.160096996 podStartE2EDuration="1.160096996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:29:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:29:54.158462922 +0000 UTC m=+1530.169701544" watchObservedRunningTime="2026-07-28 01:29:54.160096996 +0000 UTC m=+1530.171335604"
Jul 28 01:29:56.647531 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:56.647496    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:56.647994 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:56.647763    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:56.660246 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:56.660224    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:29:57.166857 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:29:57.166824    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:30:03.728463 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:03.728427    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:30:03.738035 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:03.738008    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:30:03.738157 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:03.738054    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:30:03.750463 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:03.750442    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:30:04.203270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:04.203241    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:30:20.260392 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:20.260264    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"]
Jul 28 01:30:21.813477 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:21.813447    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"]
Jul 28 01:30:24.366134 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:24.366090    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:30:35.282219 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.282155    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2" podUID="71fe1053-b099-41ff-a33d-14948f90ca37" containerName="main" containerID="cri-o://4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122" gracePeriod=45
Jul 28 01:30:35.541589 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.541528    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:30:35.643348 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643314    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71fe1053-b099-41ff-a33d-14948f90ca37-tls-certs\") pod \"71fe1053-b099-41ff-a33d-14948f90ca37\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") "
Jul 28 01:30:35.643348 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643351    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-tmp-dir\") pod \"71fe1053-b099-41ff-a33d-14948f90ca37\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") "
Jul 28 01:30:35.643608 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643383    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgshp\" (UniqueName: \"kubernetes.io/projected/71fe1053-b099-41ff-a33d-14948f90ca37-kube-api-access-jgshp\") pod \"71fe1053-b099-41ff-a33d-14948f90ca37\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") "
Jul 28 01:30:35.643608 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643512    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-home\") pod \"71fe1053-b099-41ff-a33d-14948f90ca37\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") "
Jul 28 01:30:35.643608 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643590    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-dshm\") pod \"71fe1053-b099-41ff-a33d-14948f90ca37\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") "
Jul 28 01:30:35.643763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643650    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "71fe1053-b099-41ff-a33d-14948f90ca37" (UID: "71fe1053-b099-41ff-a33d-14948f90ca37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:35.643763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643683    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-model-cache\") pod \"71fe1053-b099-41ff-a33d-14948f90ca37\" (UID: \"71fe1053-b099-41ff-a33d-14948f90ca37\") "
Jul 28 01:30:35.643763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643701    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-home" (OuterVolumeSpecName: "home") pod "71fe1053-b099-41ff-a33d-14948f90ca37" (UID: "71fe1053-b099-41ff-a33d-14948f90ca37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:35.643959 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.643932    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-model-cache" (OuterVolumeSpecName: "model-cache") pod "71fe1053-b099-41ff-a33d-14948f90ca37" (UID: "71fe1053-b099-41ff-a33d-14948f90ca37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:35.644097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.644075    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:35.644159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.644097    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:35.644159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.644110    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:35.645833 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.645801    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71fe1053-b099-41ff-a33d-14948f90ca37-kube-api-access-jgshp" (OuterVolumeSpecName: "kube-api-access-jgshp") pod "71fe1053-b099-41ff-a33d-14948f90ca37" (UID: "71fe1053-b099-41ff-a33d-14948f90ca37"). InnerVolumeSpecName "kube-api-access-jgshp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:30:35.645947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.645861    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71fe1053-b099-41ff-a33d-14948f90ca37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "71fe1053-b099-41ff-a33d-14948f90ca37" (UID: "71fe1053-b099-41ff-a33d-14948f90ca37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:30:35.645947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.645899    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-dshm" (OuterVolumeSpecName: "dshm") pod "71fe1053-b099-41ff-a33d-14948f90ca37" (UID: "71fe1053-b099-41ff-a33d-14948f90ca37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:35.745270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.745234    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71fe1053-b099-41ff-a33d-14948f90ca37-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:35.745270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.745262    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgshp\" (UniqueName: \"kubernetes.io/projected/71fe1053-b099-41ff-a33d-14948f90ca37-kube-api-access-jgshp\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:35.745270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:35.745272    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71fe1053-b099-41ff-a33d-14948f90ca37-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:36.338140 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.338104    2566 generic.go:356] "Generic (PLEG): container finished" podID="71fe1053-b099-41ff-a33d-14948f90ca37" containerID="4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122" exitCode=0
Jul 28 01:30:36.338594 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.338144    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"
Jul 28 01:30:36.338594 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.338196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2" event={"ID":"71fe1053-b099-41ff-a33d-14948f90ca37","Type":"ContainerDied","Data":"4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122"}
Jul 28 01:30:36.338594 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.338239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2" event={"ID":"71fe1053-b099-41ff-a33d-14948f90ca37","Type":"ContainerDied","Data":"c501a2796bafbec451f953d80d5ea831eb2a67cfda6bdf471eb66d33e4224455"}
Jul 28 01:30:36.338594 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.338257    2566 scope.go:117] "RemoveContainer" containerID="4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122"
Jul 28 01:30:36.347466 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.347448    2566 scope.go:117] "RemoveContainer" containerID="4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122"
Jul 28 01:30:36.347733 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:30:36.347715    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122\": container with ID starting with 4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122 not found: ID does not exist" containerID="4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122"
Jul 28 01:30:36.347774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.347743    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122"} err="failed to get container status \"4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122\": rpc error: code = NotFound desc = could not find container \"4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122\": container with ID starting with 4adcd2647ff246e65dfcbc91b5c44f1808a2d07d0dd63a07f63a031e64348122 not found: ID does not exist"
Jul 28 01:30:36.358268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.358244    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"]
Jul 28 01:30:36.363596 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.363576    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-0e04e8ff-kserve-55d9c7b767-mcxz2"]
Jul 28 01:30:36.565676 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.565640    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71fe1053-b099-41ff-a33d-14948f90ca37" path="/var/lib/kubelet/pods/71fe1053-b099-41ff-a33d-14948f90ca37/volumes"
Jul 28 01:30:36.834397 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:36.834293    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d" podUID="d01afab1-4667-45f7-8029-1dc927c3270e" containerName="main" containerID="cri-o://4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db" gracePeriod=45
Jul 28 01:30:37.084081 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.084055    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:30:37.156735 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.156702    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01afab1-4667-45f7-8029-1dc927c3270e-tls-certs\") pod \"d01afab1-4667-45f7-8029-1dc927c3270e\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") "
Jul 28 01:30:37.156735 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.156746    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-home\") pod \"d01afab1-4667-45f7-8029-1dc927c3270e\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") "
Jul 28 01:30:37.157027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.156799    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkmf8\" (UniqueName: \"kubernetes.io/projected/d01afab1-4667-45f7-8029-1dc927c3270e-kube-api-access-mkmf8\") pod \"d01afab1-4667-45f7-8029-1dc927c3270e\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") "
Jul 28 01:30:37.157027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.156841    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-model-cache\") pod \"d01afab1-4667-45f7-8029-1dc927c3270e\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") "
Jul 28 01:30:37.157027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.156907    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-tmp-dir\") pod \"d01afab1-4667-45f7-8029-1dc927c3270e\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") "
Jul 28 01:30:37.157027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.156934    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-dshm\") pod \"d01afab1-4667-45f7-8029-1dc927c3270e\" (UID: \"d01afab1-4667-45f7-8029-1dc927c3270e\") "
Jul 28 01:30:37.157233 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.157085    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-home" (OuterVolumeSpecName: "home") pod "d01afab1-4667-45f7-8029-1dc927c3270e" (UID: "d01afab1-4667-45f7-8029-1dc927c3270e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:37.157233 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.157187    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-model-cache" (OuterVolumeSpecName: "model-cache") pod "d01afab1-4667-45f7-8029-1dc927c3270e" (UID: "d01afab1-4667-45f7-8029-1dc927c3270e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:37.157233 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.157204    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d01afab1-4667-45f7-8029-1dc927c3270e" (UID: "d01afab1-4667-45f7-8029-1dc927c3270e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:37.157358 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.157238    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:37.159101 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.159076    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-dshm" (OuterVolumeSpecName: "dshm") pod "d01afab1-4667-45f7-8029-1dc927c3270e" (UID: "d01afab1-4667-45f7-8029-1dc927c3270e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:30:37.159200 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.159103    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d01afab1-4667-45f7-8029-1dc927c3270e-kube-api-access-mkmf8" (OuterVolumeSpecName: "kube-api-access-mkmf8") pod "d01afab1-4667-45f7-8029-1dc927c3270e" (UID: "d01afab1-4667-45f7-8029-1dc927c3270e"). InnerVolumeSpecName "kube-api-access-mkmf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:30:37.159200 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.159102    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d01afab1-4667-45f7-8029-1dc927c3270e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d01afab1-4667-45f7-8029-1dc927c3270e" (UID: "d01afab1-4667-45f7-8029-1dc927c3270e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:30:37.257653 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.257616    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:37.257653 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.257644    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:37.257653 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.257655    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d01afab1-4667-45f7-8029-1dc927c3270e-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:37.257653 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.257663    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkmf8\" (UniqueName: \"kubernetes.io/projected/d01afab1-4667-45f7-8029-1dc927c3270e-kube-api-access-mkmf8\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:37.257982 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.257672    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d01afab1-4667-45f7-8029-1dc927c3270e-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:30:37.345287 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.345250    2566 generic.go:356] "Generic (PLEG): container finished" podID="d01afab1-4667-45f7-8029-1dc927c3270e" containerID="4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db" exitCode=0
Jul 28 01:30:37.345685 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.345297    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"
Jul 28 01:30:37.345685 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.345311    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d" event={"ID":"d01afab1-4667-45f7-8029-1dc927c3270e","Type":"ContainerDied","Data":"4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db"}
Jul 28 01:30:37.345685 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.345343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d" event={"ID":"d01afab1-4667-45f7-8029-1dc927c3270e","Type":"ContainerDied","Data":"49fe46d87c6ac23b877081b7f4939c6258df8f0801624e792c71928d903e7023"}
Jul 28 01:30:37.345685 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.345364    2566 scope.go:117] "RemoveContainer" containerID="4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db"
Jul 28 01:30:37.354385 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.354369    2566 scope.go:117] "RemoveContainer" containerID="4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db"
Jul 28 01:30:37.354610 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:30:37.354592    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db\": container with ID starting with 4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db not found: ID does not exist" containerID="4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db"
Jul 28 01:30:37.354667 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.354618    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db"} err="failed to get container status \"4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db\": rpc error: code = NotFound desc = could not find container \"4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db\": container with ID starting with 4838868332f3247a1b1eab59fbeac5f2391bfeea997194e90d9f8fd7795329db not found: ID does not exist"
Jul 28 01:30:37.366256 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.366236    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"]
Jul 28 01:30:37.370501 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.370479    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-0e04e8ff-kserve-7dfc7f5d8c-xx69d"]
Jul 28 01:30:37.632405 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:37.632370    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:30:38.560256 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:38.560216    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d01afab1-4667-45f7-8029-1dc927c3270e" path="/var/lib/kubelet/pods/d01afab1-4667-45f7-8029-1dc927c3270e/volumes"
Jul 28 01:30:46.939675 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:46.939640    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:30:55.235476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:30:55.235443    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:31:06.541026 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:31:06.540983    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:31:16.230139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:31:16.230100    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:31:26.434949 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:31:26.434916    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:31:35.643743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:31:35.643702    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:31:46.837844 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:31:46.837806    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:31:59.440355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:31:59.440316    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:32:08.236776 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:32:08.236737    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:32:18.132071 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:32:18.132032    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:32:29.133880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:32:29.133839    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:32:39.934102 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:32:39.934061    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:32:46.436929 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:32:46.436839    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:32:55.736385 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:32:55.736347    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:33:03.641010 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:33:03.640963    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:33:15.142691 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:33:15.142640    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:33:26.045588 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:33:26.045548    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:33:39.240084 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:33:39.240041    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:33:49.530098 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:33:49.530060    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:33:58.932734 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:33:58.932694    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:34:07.033275 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:07.033243    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:34:27.207804 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.207768    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"]
Jul 28 01:34:27.208291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.208168    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71fe1053-b099-41ff-a33d-14948f90ca37" containerName="main"
Jul 28 01:34:27.208291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.208180    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="71fe1053-b099-41ff-a33d-14948f90ca37" containerName="main"
Jul 28 01:34:27.208291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.208189    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d01afab1-4667-45f7-8029-1dc927c3270e" containerName="main"
Jul 28 01:34:27.208291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.208194    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d01afab1-4667-45f7-8029-1dc927c3270e" containerName="main"
Jul 28 01:34:27.208291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.208242    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d01afab1-4667-45f7-8029-1dc927c3270e" containerName="main"
Jul 28 01:34:27.208291 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.208251    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="71fe1053-b099-41ff-a33d-14948f90ca37" containerName="main"
Jul 28 01:34:27.211134 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.211119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:27.213737 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.213703    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:34:27.213737 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.213703    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:34:27.222010 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.221961    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"]
Jul 28 01:34:27.357932 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.357896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51c4cadd-25a6-4dba-a7ed-1d672101b756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r\" (UID: \"51c4cadd-25a6-4dba-a7ed-1d672101b756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:27.459004 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.458887    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51c4cadd-25a6-4dba-a7ed-1d672101b756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r\" (UID: \"51c4cadd-25a6-4dba-a7ed-1d672101b756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:27.459529 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.459509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51c4cadd-25a6-4dba-a7ed-1d672101b756-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r\" (UID: \"51c4cadd-25a6-4dba-a7ed-1d672101b756\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:27.522698 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.522653    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:27.661544 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.661493    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"]
Jul 28 01:34:27.663722 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:34:27.663695    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c4cadd_25a6_4dba_a7ed_1d672101b756.slice/crio-c460896fe9db8ec388435d6971532948efcc48b87fb34fba37ebb954505fbaa2 WatchSource:0}: Error finding container c460896fe9db8ec388435d6971532948efcc48b87fb34fba37ebb954505fbaa2: Status 404 returned error can't find the container with id c460896fe9db8ec388435d6971532948efcc48b87fb34fba37ebb954505fbaa2
Jul 28 01:34:27.665635 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:27.665607    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:34:28.385307 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:28.385269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r" event={"ID":"51c4cadd-25a6-4dba-a7ed-1d672101b756","Type":"ContainerStarted","Data":"c460896fe9db8ec388435d6971532948efcc48b87fb34fba37ebb954505fbaa2"}
Jul 28 01:34:31.401625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:31.401585    2566 generic.go:356] "Generic (PLEG): container finished" podID="51c4cadd-25a6-4dba-a7ed-1d672101b756" containerID="48f3bb961fb82cbb4ade9001cac0c006c1000bb1823fd17ebedcd5a13e3d5582" exitCode=0
Jul 28 01:34:31.402040 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:31.401670    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r" event={"ID":"51c4cadd-25a6-4dba-a7ed-1d672101b756","Type":"ContainerDied","Data":"48f3bb961fb82cbb4ade9001cac0c006c1000bb1823fd17ebedcd5a13e3d5582"}
Jul 28 01:34:32.533292 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:32.533264    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:32.706931 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:32.706845    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51c4cadd-25a6-4dba-a7ed-1d672101b756-maas-api-service-ca\") pod \"51c4cadd-25a6-4dba-a7ed-1d672101b756\" (UID: \"51c4cadd-25a6-4dba-a7ed-1d672101b756\") "
Jul 28 01:34:32.707237 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:32.707214    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51c4cadd-25a6-4dba-a7ed-1d672101b756-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51c4cadd-25a6-4dba-a7ed-1d672101b756" (UID: "51c4cadd-25a6-4dba-a7ed-1d672101b756"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:34:32.808318 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:32.808281    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51c4cadd-25a6-4dba-a7ed-1d672101b756-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:34:33.411457 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:33.411428    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"
Jul 28 01:34:33.411457 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:33.411453    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r" event={"ID":"51c4cadd-25a6-4dba-a7ed-1d672101b756","Type":"ContainerDied","Data":"c460896fe9db8ec388435d6971532948efcc48b87fb34fba37ebb954505fbaa2"}
Jul 28 01:34:33.411670 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:33.411497    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c460896fe9db8ec388435d6971532948efcc48b87fb34fba37ebb954505fbaa2"
Jul 28 01:34:42.845211 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:42.845179    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"]
Jul 28 01:34:42.850014 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:42.849983    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5ce185b9-6bp9r"]
Jul 28 01:34:44.560148 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:44.560118    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51c4cadd-25a6-4dba-a7ed-1d672101b756" path="/var/lib/kubelet/pods/51c4cadd-25a6-4dba-a7ed-1d672101b756/volumes"
Jul 28 01:34:46.337278 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:46.337241    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:34:47.947775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.947744    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"]
Jul 28 01:34:47.948217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.948109    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51c4cadd-25a6-4dba-a7ed-1d672101b756" containerName="revoke-keys"
Jul 28 01:34:47.948217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.948122    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="51c4cadd-25a6-4dba-a7ed-1d672101b756" containerName="revoke-keys"
Jul 28 01:34:47.948217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.948177    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="51c4cadd-25a6-4dba-a7ed-1d672101b756" containerName="revoke-keys"
Jul 28 01:34:47.951200 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.951181    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:47.953689 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.953667    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b70cb0d5-dockercfg-9nfr2\""
Jul 28 01:34:47.953843 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.953825    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b70cb0d5\""
Jul 28 01:34:47.960239 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:47.960219    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"]
Jul 28 01:34:48.008932 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.008897    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"]
Jul 28 01:34:48.012109 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.012091    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.026598 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.026572    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"]
Jul 28 01:34:48.033114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.033086    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k42dn\" (UniqueName: \"kubernetes.io/projected/337ca727-863f-4225-89d8-5beb1b4517f7-kube-api-access-k42dn\") pod \"payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") " pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.033263 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.033120    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdq24\" (UniqueName: \"kubernetes.io/projected/70228031-3e33-4a08-9e08-21651ef33a9d-kube-api-access-fdq24\") pod \"payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.033263 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.033146    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ca727-863f-4225-89d8-5beb1b4517f7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") " pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.033263 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.033238    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70228031-3e33-4a08-9e08-21651ef33a9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.134617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.134568    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k42dn\" (UniqueName: \"kubernetes.io/projected/337ca727-863f-4225-89d8-5beb1b4517f7-kube-api-access-k42dn\") pod \"payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") " pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.134617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.134619    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdq24\" (UniqueName: \"kubernetes.io/projected/70228031-3e33-4a08-9e08-21651ef33a9d-kube-api-access-fdq24\") pod \"payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.134891 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.134650    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ca727-863f-4225-89d8-5beb1b4517f7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") " pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.134891 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.134689    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70228031-3e33-4a08-9e08-21651ef33a9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.135522 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.135498    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ca727-863f-4225-89d8-5beb1b4517f7-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") " pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.135608 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.135524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70228031-3e33-4a08-9e08-21651ef33a9d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.143573 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.143538    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k42dn\" (UniqueName: \"kubernetes.io/projected/337ca727-863f-4225-89d8-5beb1b4517f7-kube-api-access-k42dn\") pod \"payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") " pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.143652 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.143588    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdq24\" (UniqueName: \"kubernetes.io/projected/70228031-3e33-4a08-9e08-21651ef33a9d-kube-api-access-fdq24\") pod \"payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.262359 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.262269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:48.321710 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.321678    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:48.416900 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.416862    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"]
Jul 28 01:34:48.419461 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:34:48.419424    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337ca727_863f_4225_89d8_5beb1b4517f7.slice/crio-392e968195c638f23132647bdab5afcb86f416dce71e8ea62b329590192ff9a6 WatchSource:0}: Error finding container 392e968195c638f23132647bdab5afcb86f416dce71e8ea62b329590192ff9a6: Status 404 returned error can't find the container with id 392e968195c638f23132647bdab5afcb86f416dce71e8ea62b329590192ff9a6
Jul 28 01:34:48.475831 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.475803    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"]
Jul 28 01:34:48.477012 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:34:48.476981    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70228031_3e33_4a08_9e08_21651ef33a9d.slice/crio-296e5000141098b95748d87658c0f9f523013477378066cc294167265c313202 WatchSource:0}: Error finding container 296e5000141098b95748d87658c0f9f523013477378066cc294167265c313202: Status 404 returned error can't find the container with id 296e5000141098b95748d87658c0f9f523013477378066cc294167265c313202
Jul 28 01:34:48.482040 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:48.482012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp" event={"ID":"337ca727-863f-4225-89d8-5beb1b4517f7","Type":"ContainerStarted","Data":"392e968195c638f23132647bdab5afcb86f416dce71e8ea62b329590192ff9a6"}
Jul 28 01:34:49.491785 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:49.491724    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g" event={"ID":"70228031-3e33-4a08-9e08-21651ef33a9d","Type":"ContainerStarted","Data":"296e5000141098b95748d87658c0f9f523013477378066cc294167265c313202"}
Jul 28 01:34:50.498017 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:50.497902    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp" event={"ID":"337ca727-863f-4225-89d8-5beb1b4517f7","Type":"ContainerStarted","Data":"0fb7c1705054d57a25a735ae57bef933cd6d5dc8befc7f1ddffff30acb25beb7"}
Jul 28 01:34:50.498017 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:50.497991    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:34:50.499407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:50.499377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g" event={"ID":"70228031-3e33-4a08-9e08-21651ef33a9d","Type":"ContainerStarted","Data":"5749ec04ffab338594be6f29deb5852c3aaa1de68eceb5ba3c6f6aca6c8d7bb3"}
Jul 28 01:34:50.499512 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:50.499430    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:34:50.518483 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:50.518429    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp" podStartSLOduration=1.729173031 podStartE2EDuration="3.518410876s" totalImagesPullingTime="1.789237845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:34:50.516959655 +0000 UTC m=+1826.528198279" watchObservedRunningTime="2026-07-28 01:34:50.518410876 +0000 UTC m=+1826.529649485"
Jul 28 01:34:50.537292 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:50.537234    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g" podStartSLOduration=1.798410428 podStartE2EDuration="3.537217992s" totalImagesPullingTime="1.738807564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:34:50.535806024 +0000 UTC m=+1826.547044633" watchObservedRunningTime="2026-07-28 01:34:50.537217992 +0000 UTC m=+1826.548456662"
Jul 28 01:34:55.732275 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:34:55.732240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:35:01.506865 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:01.506830    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:35:01.507366 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:01.506901    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:35:11.924833 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:11.924788    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"]
Jul 28 01:35:11.930479 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:11.930454    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:11.933281 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:11.933262    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:35:11.933412 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:11.933267    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:35:11.937058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:11.937033    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"]
Jul 28 01:35:12.055833 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.055793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf\" (UID: \"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:12.157037 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.156997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf\" (UID: \"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:12.157602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.157580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf\" (UID: \"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:12.242500 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.242416    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:12.379009 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.378957    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"]
Jul 28 01:35:12.381172 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:35:12.381144    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbc0ef4_707a_4aa6_8883_59cb8f5dc327.slice/crio-4bc78b09af85d8fe497657fdf3ca3b207df454a14307d3ff114b7ebebc562648 WatchSource:0}: Error finding container 4bc78b09af85d8fe497657fdf3ca3b207df454a14307d3ff114b7ebebc562648: Status 404 returned error can't find the container with id 4bc78b09af85d8fe497657fdf3ca3b207df454a14307d3ff114b7ebebc562648
Jul 28 01:35:12.609572 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.609536    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf" event={"ID":"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327","Type":"ContainerStarted","Data":"28ec5de47e78b9503e2a7e12a96e7a1102b7f8ffe48cc309679c390fc093d6e5"}
Jul 28 01:35:12.609742 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.609590    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf" event={"ID":"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327","Type":"ContainerStarted","Data":"4bc78b09af85d8fe497657fdf3ca3b207df454a14307d3ff114b7ebebc562648"}
Jul 28 01:35:12.628226 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:12.628169    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf" podStartSLOduration=1.6281520299999999 podStartE2EDuration="1.62815203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:12.625279315 +0000 UTC m=+1848.636517924" watchObservedRunningTime="2026-07-28 01:35:12.62815203 +0000 UTC m=+1848.639390644"
Jul 28 01:35:13.616150 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:13.616116    2566 generic.go:356] "Generic (PLEG): container finished" podID="bcbc0ef4-707a-4aa6-8883-59cb8f5dc327" containerID="28ec5de47e78b9503e2a7e12a96e7a1102b7f8ffe48cc309679c390fc093d6e5" exitCode=0
Jul 28 01:35:13.616612 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:13.616170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf" event={"ID":"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327","Type":"ContainerDied","Data":"28ec5de47e78b9503e2a7e12a96e7a1102b7f8ffe48cc309679c390fc093d6e5"}
Jul 28 01:35:14.750329 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:14.750305    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:14.880366 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:14.880266    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327-maas-api-service-ca\") pod \"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327\" (UID: \"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327\") "
Jul 28 01:35:14.880636 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:14.880614    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bcbc0ef4-707a-4aa6-8883-59cb8f5dc327" (UID: "bcbc0ef4-707a-4aa6-8883-59cb8f5dc327"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:14.980960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:14.980924    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:15.626505 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:15.626471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf" event={"ID":"bcbc0ef4-707a-4aa6-8883-59cb8f5dc327","Type":"ContainerDied","Data":"4bc78b09af85d8fe497657fdf3ca3b207df454a14307d3ff114b7ebebc562648"}
Jul 28 01:35:15.626505 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:15.626493    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"
Jul 28 01:35:15.626715 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:15.626520    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bc78b09af85d8fe497657fdf3ca3b207df454a14307d3ff114b7ebebc562648"
Jul 28 01:35:18.513325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.513289    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"]
Jul 28 01:35:18.513802 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.513547    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp" podUID="337ca727-863f-4225-89d8-5beb1b4517f7" containerName="payload-processing" containerID="cri-o://0fb7c1705054d57a25a735ae57bef933cd6d5dc8befc7f1ddffff30acb25beb7" gracePeriod=30
Jul 28 01:35:18.523552 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.523511    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"]
Jul 28 01:35:18.523788 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.523759    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g" podUID="70228031-3e33-4a08-9e08-21651ef33a9d" containerName="payload-pre-processing" containerID="cri-o://5749ec04ffab338594be6f29deb5852c3aaa1de68eceb5ba3c6f6aca6c8d7bb3" gracePeriod=30
Jul 28 01:35:18.642114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.642085    2566 generic.go:356] "Generic (PLEG): container finished" podID="337ca727-863f-4225-89d8-5beb1b4517f7" containerID="0fb7c1705054d57a25a735ae57bef933cd6d5dc8befc7f1ddffff30acb25beb7" exitCode=0
Jul 28 01:35:18.642264 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.642166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp" event={"ID":"337ca727-863f-4225-89d8-5beb1b4517f7","Type":"ContainerDied","Data":"0fb7c1705054d57a25a735ae57bef933cd6d5dc8befc7f1ddffff30acb25beb7"}
Jul 28 01:35:18.643878 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.643854    2566 generic.go:356] "Generic (PLEG): container finished" podID="70228031-3e33-4a08-9e08-21651ef33a9d" containerID="5749ec04ffab338594be6f29deb5852c3aaa1de68eceb5ba3c6f6aca6c8d7bb3" exitCode=0
Jul 28 01:35:18.644019 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.643946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g" event={"ID":"70228031-3e33-4a08-9e08-21651ef33a9d","Type":"ContainerDied","Data":"5749ec04ffab338594be6f29deb5852c3aaa1de68eceb5ba3c6f6aca6c8d7bb3"}
Jul 28 01:35:18.663713 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.663567    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"]
Jul 28 01:35:18.665314 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.665291    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-b70cb0d5-g8mdf"]
Jul 28 01:35:18.800225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.800195    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:35:18.805898 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.805873    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:35:18.913237 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.913197    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdq24\" (UniqueName: \"kubernetes.io/projected/70228031-3e33-4a08-9e08-21651ef33a9d-kube-api-access-fdq24\") pod \"70228031-3e33-4a08-9e08-21651ef33a9d\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") "
Jul 28 01:35:18.913433 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.913274    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70228031-3e33-4a08-9e08-21651ef33a9d-plugins-config-volume\") pod \"70228031-3e33-4a08-9e08-21651ef33a9d\" (UID: \"70228031-3e33-4a08-9e08-21651ef33a9d\") "
Jul 28 01:35:18.913433 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.913299    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k42dn\" (UniqueName: \"kubernetes.io/projected/337ca727-863f-4225-89d8-5beb1b4517f7-kube-api-access-k42dn\") pod \"337ca727-863f-4225-89d8-5beb1b4517f7\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") "
Jul 28 01:35:18.913433 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.913373    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ca727-863f-4225-89d8-5beb1b4517f7-plugins-config-volume\") pod \"337ca727-863f-4225-89d8-5beb1b4517f7\" (UID: \"337ca727-863f-4225-89d8-5beb1b4517f7\") "
Jul 28 01:35:18.913811 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.913774    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70228031-3e33-4a08-9e08-21651ef33a9d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "70228031-3e33-4a08-9e08-21651ef33a9d" (UID: "70228031-3e33-4a08-9e08-21651ef33a9d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:18.914143 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.913811    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337ca727-863f-4225-89d8-5beb1b4517f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "337ca727-863f-4225-89d8-5beb1b4517f7" (UID: "337ca727-863f-4225-89d8-5beb1b4517f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:18.915592 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.915567    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337ca727-863f-4225-89d8-5beb1b4517f7-kube-api-access-k42dn" (OuterVolumeSpecName: "kube-api-access-k42dn") pod "337ca727-863f-4225-89d8-5beb1b4517f7" (UID: "337ca727-863f-4225-89d8-5beb1b4517f7"). InnerVolumeSpecName "kube-api-access-k42dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:18.915926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:18.915903    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70228031-3e33-4a08-9e08-21651ef33a9d-kube-api-access-fdq24" (OuterVolumeSpecName: "kube-api-access-fdq24") pod "70228031-3e33-4a08-9e08-21651ef33a9d" (UID: "70228031-3e33-4a08-9e08-21651ef33a9d"). InnerVolumeSpecName "kube-api-access-fdq24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:19.014159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.014121    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdq24\" (UniqueName: \"kubernetes.io/projected/70228031-3e33-4a08-9e08-21651ef33a9d-kube-api-access-fdq24\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:19.014159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.014149    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70228031-3e33-4a08-9e08-21651ef33a9d-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:19.014159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.014160    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k42dn\" (UniqueName: \"kubernetes.io/projected/337ca727-863f-4225-89d8-5beb1b4517f7-kube-api-access-k42dn\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:19.014159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.014168    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337ca727-863f-4225-89d8-5beb1b4517f7-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:19.651671 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.651639    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"
Jul 28 01:35:19.652176 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.651667    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp" event={"ID":"337ca727-863f-4225-89d8-5beb1b4517f7","Type":"ContainerDied","Data":"392e968195c638f23132647bdab5afcb86f416dce71e8ea62b329590192ff9a6"}
Jul 28 01:35:19.652176 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.651744    2566 scope.go:117] "RemoveContainer" containerID="0fb7c1705054d57a25a735ae57bef933cd6d5dc8befc7f1ddffff30acb25beb7"
Jul 28 01:35:19.653215 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.653195    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"
Jul 28 01:35:19.653215 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.653208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g" event={"ID":"70228031-3e33-4a08-9e08-21651ef33a9d","Type":"ContainerDied","Data":"296e5000141098b95748d87658c0f9f523013477378066cc294167265c313202"}
Jul 28 01:35:19.662558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.662539    2566 scope.go:117] "RemoveContainer" containerID="5749ec04ffab338594be6f29deb5852c3aaa1de68eceb5ba3c6f6aca6c8d7bb3"
Jul 28 01:35:19.676938 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.676912    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"]
Jul 28 01:35:19.681278 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.681253    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b70cb0d5-5cf4fc6bf7-jj7qp"]
Jul 28 01:35:19.693595 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.693556    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"]
Jul 28 01:35:19.700699 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:19.700667    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b70cb0d5-85bf78fdd4-b5l7g"]
Jul 28 01:35:20.560577 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:20.560536    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="337ca727-863f-4225-89d8-5beb1b4517f7" path="/var/lib/kubelet/pods/337ca727-863f-4225-89d8-5beb1b4517f7/volumes"
Jul 28 01:35:20.561800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:20.560952    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70228031-3e33-4a08-9e08-21651ef33a9d" path="/var/lib/kubelet/pods/70228031-3e33-4a08-9e08-21651ef33a9d/volumes"
Jul 28 01:35:20.561800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:20.561291    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcbc0ef4-707a-4aa6-8883-59cb8f5dc327" path="/var/lib/kubelet/pods/bcbc0ef4-707a-4aa6-8883-59cb8f5dc327/volumes"
Jul 28 01:35:22.340789 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:22.340747    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:35:23.005700 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.005653    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"]
Jul 28 01:35:23.006127 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006099    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="337ca727-863f-4225-89d8-5beb1b4517f7" containerName="payload-processing"
Jul 28 01:35:23.006127 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006125    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="337ca727-863f-4225-89d8-5beb1b4517f7" containerName="payload-processing"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006147    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcbc0ef4-707a-4aa6-8883-59cb8f5dc327" containerName="revoke-keys"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006156    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcbc0ef4-707a-4aa6-8883-59cb8f5dc327" containerName="revoke-keys"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006171    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70228031-3e33-4a08-9e08-21651ef33a9d" containerName="payload-pre-processing"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006179    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70228031-3e33-4a08-9e08-21651ef33a9d" containerName="payload-pre-processing"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006262    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcbc0ef4-707a-4aa6-8883-59cb8f5dc327" containerName="revoke-keys"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006275    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="337ca727-863f-4225-89d8-5beb1b4517f7" containerName="payload-processing"
Jul 28 01:35:23.006410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.006287    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70228031-3e33-4a08-9e08-21651ef33a9d" containerName="payload-pre-processing"
Jul 28 01:35:23.008586 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.008569    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.011236 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.011214    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0fcf74cb-dockercfg-g8rdq\""
Jul 28 01:35:23.011461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.011444    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0fcf74cb\""
Jul 28 01:35:23.021127 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.021098    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"]
Jul 28 01:35:23.068054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.068019    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"]
Jul 28 01:35:23.070276 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.070257    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.087804 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.087774    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"]
Jul 28 01:35:23.151578 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.151538    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/449d87cc-9cc0-4648-bef0-fb406761762c-kube-api-access-5mqj7\") pod \"payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.151775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.151610    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgn2n\" (UniqueName: \"kubernetes.io/projected/1cf38388-d942-4962-ab3d-c6ef572633a8-kube-api-access-cgn2n\") pod \"payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") " pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.151775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.151639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449d87cc-9cc0-4648-bef0-fb406761762c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.151775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.151680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf38388-d942-4962-ab3d-c6ef572633a8-plugins-config-volume\") pod \"payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") " pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.252925 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.252882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/449d87cc-9cc0-4648-bef0-fb406761762c-kube-api-access-5mqj7\") pod \"payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.253120 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.252952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgn2n\" (UniqueName: \"kubernetes.io/projected/1cf38388-d942-4962-ab3d-c6ef572633a8-kube-api-access-cgn2n\") pod \"payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") " pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.253120 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.253021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449d87cc-9cc0-4648-bef0-fb406761762c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.253120 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.253067    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf38388-d942-4962-ab3d-c6ef572633a8-plugins-config-volume\") pod \"payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") " pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.253807 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.253786    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449d87cc-9cc0-4648-bef0-fb406761762c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.254086 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.254068    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf38388-d942-4962-ab3d-c6ef572633a8-plugins-config-volume\") pod \"payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") " pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.261453 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.261396    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/449d87cc-9cc0-4648-bef0-fb406761762c-kube-api-access-5mqj7\") pod \"payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.261560 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.261493    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgn2n\" (UniqueName: \"kubernetes.io/projected/1cf38388-d942-4962-ab3d-c6ef572633a8-kube-api-access-cgn2n\") pod \"payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") " pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.319782 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.319737    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.380348 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.380092    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.467305 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.467275    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"]
Jul 28 01:35:23.468873 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:35:23.468842    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf38388_d942_4962_ab3d_c6ef572633a8.slice/crio-7455864d723afd45508258a61876dda00b5fbfd1fc5d6ef61dcebea07846bab1 WatchSource:0}: Error finding container 7455864d723afd45508258a61876dda00b5fbfd1fc5d6ef61dcebea07846bab1: Status 404 returned error can't find the container with id 7455864d723afd45508258a61876dda00b5fbfd1fc5d6ef61dcebea07846bab1
Jul 28 01:35:23.533108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.533041    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"]
Jul 28 01:35:23.535368 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:35:23.535334    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449d87cc_9cc0_4648_bef0_fb406761762c.slice/crio-5689f941a6941eb834cc1e5373a8e5411993015946813ea836068474b41c0bf9 WatchSource:0}: Error finding container 5689f941a6941eb834cc1e5373a8e5411993015946813ea836068474b41c0bf9: Status 404 returned error can't find the container with id 5689f941a6941eb834cc1e5373a8e5411993015946813ea836068474b41c0bf9
Jul 28 01:35:23.676001 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.675926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h" event={"ID":"1cf38388-d942-4962-ab3d-c6ef572633a8","Type":"ContainerStarted","Data":"50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431"}
Jul 28 01:35:23.676198 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.676029    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:23.676198 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.676055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h" event={"ID":"1cf38388-d942-4962-ab3d-c6ef572633a8","Type":"ContainerStarted","Data":"7455864d723afd45508258a61876dda00b5fbfd1fc5d6ef61dcebea07846bab1"}
Jul 28 01:35:23.677631 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.677602    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97" event={"ID":"449d87cc-9cc0-4648-bef0-fb406761762c","Type":"ContainerStarted","Data":"f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8"}
Jul 28 01:35:23.677727 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.677653    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97" event={"ID":"449d87cc-9cc0-4648-bef0-fb406761762c","Type":"ContainerStarted","Data":"5689f941a6941eb834cc1e5373a8e5411993015946813ea836068474b41c0bf9"}
Jul 28 01:35:23.677773 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.677736    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:23.694507 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.694443    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h" podStartSLOduration=1.694425354 podStartE2EDuration="1.694425354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:23.692862467 +0000 UTC m=+1859.704101086" watchObservedRunningTime="2026-07-28 01:35:23.694425354 +0000 UTC m=+1859.705663963"
Jul 28 01:35:23.708584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:23.708521    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97" podStartSLOduration=0.708500493 podStartE2EDuration="708.500493ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:23.707785389 +0000 UTC m=+1859.719024018" watchObservedRunningTime="2026-07-28 01:35:23.708500493 +0000 UTC m=+1859.719739103"
Jul 28 01:35:32.535664 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:32.535630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:35:34.686107 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:34.686073    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:34.686697 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:34.686163    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:47.797044 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.796946    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"]
Jul 28 01:35:47.801550 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.801528    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:47.804240 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.804205    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:35:47.804382 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.804320    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:35:47.807511 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.807486    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"]
Jul 28 01:35:47.866023 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.865991    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37fa294-1275-45bb-a6f0-e1e5d7ebb682-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5\" (UID: \"b37fa294-1275-45bb-a6f0-e1e5d7ebb682\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:47.967496 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.967458    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37fa294-1275-45bb-a6f0-e1e5d7ebb682-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5\" (UID: \"b37fa294-1275-45bb-a6f0-e1e5d7ebb682\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:47.968128 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:47.968109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37fa294-1275-45bb-a6f0-e1e5d7ebb682-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5\" (UID: \"b37fa294-1275-45bb-a6f0-e1e5d7ebb682\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:48.112862 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:48.112813    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:48.246483 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:48.246445    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"]
Jul 28 01:35:48.249456 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:35:48.249418    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37fa294_1275_45bb_a6f0_e1e5d7ebb682.slice/crio-51606b2574778e44a0b5874505f128a7dd33e8aa8f5502a3254f9826605a5602 WatchSource:0}: Error finding container 51606b2574778e44a0b5874505f128a7dd33e8aa8f5502a3254f9826605a5602: Status 404 returned error can't find the container with id 51606b2574778e44a0b5874505f128a7dd33e8aa8f5502a3254f9826605a5602
Jul 28 01:35:48.801289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:48.801199    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5" event={"ID":"b37fa294-1275-45bb-a6f0-e1e5d7ebb682","Type":"ContainerStarted","Data":"f8412ed4ceaecd279a87dbf76b73676879cd3c42c60c799a6bbc48c83bace06e"}
Jul 28 01:35:48.801289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:48.801251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5" event={"ID":"b37fa294-1275-45bb-a6f0-e1e5d7ebb682","Type":"ContainerStarted","Data":"51606b2574778e44a0b5874505f128a7dd33e8aa8f5502a3254f9826605a5602"}
Jul 28 01:35:48.817601 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:48.817545    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5" podStartSLOduration=1.8175275910000002 podStartE2EDuration="1.817527591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:48.81627713 +0000 UTC m=+1884.827515763" watchObservedRunningTime="2026-07-28 01:35:48.817527591 +0000 UTC m=+1884.828766271"
Jul 28 01:35:49.807036 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:49.806998    2566 generic.go:356] "Generic (PLEG): container finished" podID="b37fa294-1275-45bb-a6f0-e1e5d7ebb682" containerID="f8412ed4ceaecd279a87dbf76b73676879cd3c42c60c799a6bbc48c83bace06e" exitCode=0
Jul 28 01:35:49.807431 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:49.807093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5" event={"ID":"b37fa294-1275-45bb-a6f0-e1e5d7ebb682","Type":"ContainerDied","Data":"f8412ed4ceaecd279a87dbf76b73676879cd3c42c60c799a6bbc48c83bace06e"}
Jul 28 01:35:50.940525 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:50.940500    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:51.014069 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:51.014023    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37fa294-1275-45bb-a6f0-e1e5d7ebb682-maas-api-service-ca\") pod \"b37fa294-1275-45bb-a6f0-e1e5d7ebb682\" (UID: \"b37fa294-1275-45bb-a6f0-e1e5d7ebb682\") "
Jul 28 01:35:51.014396 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:51.014371    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b37fa294-1275-45bb-a6f0-e1e5d7ebb682-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b37fa294-1275-45bb-a6f0-e1e5d7ebb682" (UID: "b37fa294-1275-45bb-a6f0-e1e5d7ebb682"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:51.115565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:51.115524    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37fa294-1275-45bb-a6f0-e1e5d7ebb682-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:51.817163 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:51.817127    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"
Jul 28 01:35:51.817339 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:51.817165    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5" event={"ID":"b37fa294-1275-45bb-a6f0-e1e5d7ebb682","Type":"ContainerDied","Data":"51606b2574778e44a0b5874505f128a7dd33e8aa8f5502a3254f9826605a5602"}
Jul 28 01:35:51.817339 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:51.817214    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51606b2574778e44a0b5874505f128a7dd33e8aa8f5502a3254f9826605a5602"
Jul 28 01:35:52.884169 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:52.884130    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"]
Jul 28 01:35:52.884929 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:52.884349    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h" podUID="1cf38388-d942-4962-ab3d-c6ef572633a8" containerName="payload-processing" containerID="cri-o://50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431" gracePeriod=30
Jul 28 01:35:52.890213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:52.889739    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"]
Jul 28 01:35:52.890213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:52.889990    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97" podUID="449d87cc-9cc0-4648-bef0-fb406761762c" containerName="payload-pre-processing" containerID="cri-o://f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8" gracePeriod=30
Jul 28 01:35:53.033258 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.033200    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"]
Jul 28 01:35:53.035015 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.034958    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-0fcf74cb-zk4x5"]
Jul 28 01:35:53.139951 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.139881    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:53.167219 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.167195    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:53.233872 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.233839    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/449d87cc-9cc0-4648-bef0-fb406761762c-kube-api-access-5mqj7\") pod \"449d87cc-9cc0-4648-bef0-fb406761762c\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") "
Jul 28 01:35:53.233872 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.233879    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449d87cc-9cc0-4648-bef0-fb406761762c-plugins-config-volume\") pod \"449d87cc-9cc0-4648-bef0-fb406761762c\" (UID: \"449d87cc-9cc0-4648-bef0-fb406761762c\") "
Jul 28 01:35:53.234149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.233903    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf38388-d942-4962-ab3d-c6ef572633a8-plugins-config-volume\") pod \"1cf38388-d942-4962-ab3d-c6ef572633a8\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") "
Jul 28 01:35:53.234149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.233939    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgn2n\" (UniqueName: \"kubernetes.io/projected/1cf38388-d942-4962-ab3d-c6ef572633a8-kube-api-access-cgn2n\") pod \"1cf38388-d942-4962-ab3d-c6ef572633a8\" (UID: \"1cf38388-d942-4962-ab3d-c6ef572633a8\") "
Jul 28 01:35:53.234294 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.234266    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/449d87cc-9cc0-4648-bef0-fb406761762c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "449d87cc-9cc0-4648-bef0-fb406761762c" (UID: "449d87cc-9cc0-4648-bef0-fb406761762c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:53.234371 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.234331    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf38388-d942-4962-ab3d-c6ef572633a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1cf38388-d942-4962-ab3d-c6ef572633a8" (UID: "1cf38388-d942-4962-ab3d-c6ef572633a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:35:53.236254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.236227    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449d87cc-9cc0-4648-bef0-fb406761762c-kube-api-access-5mqj7" (OuterVolumeSpecName: "kube-api-access-5mqj7") pod "449d87cc-9cc0-4648-bef0-fb406761762c" (UID: "449d87cc-9cc0-4648-bef0-fb406761762c"). InnerVolumeSpecName "kube-api-access-5mqj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:53.236254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.236227    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf38388-d942-4962-ab3d-c6ef572633a8-kube-api-access-cgn2n" (OuterVolumeSpecName: "kube-api-access-cgn2n") pod "1cf38388-d942-4962-ab3d-c6ef572633a8" (UID: "1cf38388-d942-4962-ab3d-c6ef572633a8"). InnerVolumeSpecName "kube-api-access-cgn2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:35:53.334768 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.334724    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgn2n\" (UniqueName: \"kubernetes.io/projected/1cf38388-d942-4962-ab3d-c6ef572633a8-kube-api-access-cgn2n\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.334768 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.334759    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mqj7\" (UniqueName: \"kubernetes.io/projected/449d87cc-9cc0-4648-bef0-fb406761762c-kube-api-access-5mqj7\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.334768 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.334769    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/449d87cc-9cc0-4648-bef0-fb406761762c-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.334768 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.334778    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1cf38388-d942-4962-ab3d-c6ef572633a8-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:35:53.829083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.829045    2566 generic.go:356] "Generic (PLEG): container finished" podID="449d87cc-9cc0-4648-bef0-fb406761762c" containerID="f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8" exitCode=0
Jul 28 01:35:53.829267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.829097    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"
Jul 28 01:35:53.829267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.829131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97" event={"ID":"449d87cc-9cc0-4648-bef0-fb406761762c","Type":"ContainerDied","Data":"f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8"}
Jul 28 01:35:53.829267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.829183    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97" event={"ID":"449d87cc-9cc0-4648-bef0-fb406761762c","Type":"ContainerDied","Data":"5689f941a6941eb834cc1e5373a8e5411993015946813ea836068474b41c0bf9"}
Jul 28 01:35:53.829267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.829213    2566 scope.go:117] "RemoveContainer" containerID="f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8"
Jul 28 01:35:53.830494 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.830468    2566 generic.go:356] "Generic (PLEG): container finished" podID="1cf38388-d942-4962-ab3d-c6ef572633a8" containerID="50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431" exitCode=0
Jul 28 01:35:53.830613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.830498    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"
Jul 28 01:35:53.830613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.830529    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h" event={"ID":"1cf38388-d942-4962-ab3d-c6ef572633a8","Type":"ContainerDied","Data":"50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431"}
Jul 28 01:35:53.830613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.830573    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h" event={"ID":"1cf38388-d942-4962-ab3d-c6ef572633a8","Type":"ContainerDied","Data":"7455864d723afd45508258a61876dda00b5fbfd1fc5d6ef61dcebea07846bab1"}
Jul 28 01:35:53.838761 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.838740    2566 scope.go:117] "RemoveContainer" containerID="f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8"
Jul 28 01:35:53.839075 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:35:53.839056    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8\": container with ID starting with f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8 not found: ID does not exist" containerID="f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8"
Jul 28 01:35:53.839150 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.839082    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8"} err="failed to get container status \"f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8\": rpc error: code = NotFound desc = could not find container \"f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8\": container with ID starting with f0e2058b431a1ada8b611d2227c564be6ecd5dd72104ebd039bcada0597a3cb8 not found: ID does not exist"
Jul 28 01:35:53.839150 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.839097    2566 scope.go:117] "RemoveContainer" containerID="50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431"
Jul 28 01:35:53.847158 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.847141    2566 scope.go:117] "RemoveContainer" containerID="50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431"
Jul 28 01:35:53.847491 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:35:53.847470    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431\": container with ID starting with 50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431 not found: ID does not exist" containerID="50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431"
Jul 28 01:35:53.847576 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.847496    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431"} err="failed to get container status \"50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431\": rpc error: code = NotFound desc = could not find container \"50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431\": container with ID starting with 50d09f6050daa665a7cb606060d2a7b2892f99f136406373e053cff01444d431 not found: ID does not exist"
Jul 28 01:35:53.853277 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.853251    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"]
Jul 28 01:35:53.857730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.857708    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-0fcf74cb-6cbc79c89d-4gx2h"]
Jul 28 01:35:53.868691 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.868664    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"]
Jul 28 01:35:53.874691 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:53.874664    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0fcf74cb-7467864b7b-9st97"]
Jul 28 01:35:54.560799 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:54.560769    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cf38388-d942-4962-ab3d-c6ef572633a8" path="/var/lib/kubelet/pods/1cf38388-d942-4962-ab3d-c6ef572633a8/volumes"
Jul 28 01:35:54.561187 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:54.561130    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="449d87cc-9cc0-4648-bef0-fb406761762c" path="/var/lib/kubelet/pods/449d87cc-9cc0-4648-bef0-fb406761762c/volumes"
Jul 28 01:35:54.561405 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:54.561392    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b37fa294-1275-45bb-a6f0-e1e5d7ebb682" path="/var/lib/kubelet/pods/b37fa294-1275-45bb-a6f0-e1e5d7ebb682/volumes"
Jul 28 01:35:56.540371 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:56.540330    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:35:58.141848 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.141813    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"]
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142149    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="449d87cc-9cc0-4648-bef0-fb406761762c" containerName="payload-pre-processing"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142160    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="449d87cc-9cc0-4648-bef0-fb406761762c" containerName="payload-pre-processing"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142170    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cf38388-d942-4962-ab3d-c6ef572633a8" containerName="payload-processing"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142176    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cf38388-d942-4962-ab3d-c6ef572633a8" containerName="payload-processing"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142186    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37fa294-1275-45bb-a6f0-e1e5d7ebb682" containerName="revoke-keys"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142193    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37fa294-1275-45bb-a6f0-e1e5d7ebb682" containerName="revoke-keys"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142247    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cf38388-d942-4962-ab3d-c6ef572633a8" containerName="payload-processing"
Jul 28 01:35:58.142255 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142256    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="449d87cc-9cc0-4648-bef0-fb406761762c" containerName="payload-pre-processing"
Jul 28 01:35:58.142502 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.142262    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37fa294-1275-45bb-a6f0-e1e5d7ebb682" containerName="revoke-keys"
Jul 28 01:35:58.146391 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.146368    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.148793 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.148772    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-30dd5a44\""
Jul 28 01:35:58.148793 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.148787    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-30dd5a44-dockercfg-c7zd4\""
Jul 28 01:35:58.156739 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.156715    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"]
Jul 28 01:35:58.158433 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.158412    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"]
Jul 28 01:35:58.161558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.161541    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.175799 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.175768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"]
Jul 28 01:35:58.276995 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.276929    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n88gj\" (UniqueName: \"kubernetes.io/projected/9606559d-a2e2-41f0-be90-b8c21e644330-kube-api-access-n88gj\") pod \"payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.277196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.277048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-524kv\" (UniqueName: \"kubernetes.io/projected/2f3b7026-2584-43c2-b6e4-084c6a40cdab-kube-api-access-524kv\") pod \"payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") " pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.277196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.277114    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f3b7026-2584-43c2-b6e4-084c6a40cdab-plugins-config-volume\") pod \"payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") " pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.277196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.277159    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9606559d-a2e2-41f0-be90-b8c21e644330-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.378548 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.378498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-524kv\" (UniqueName: \"kubernetes.io/projected/2f3b7026-2584-43c2-b6e4-084c6a40cdab-kube-api-access-524kv\") pod \"payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") " pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.378759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.378566    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f3b7026-2584-43c2-b6e4-084c6a40cdab-plugins-config-volume\") pod \"payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") " pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.378759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.378601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9606559d-a2e2-41f0-be90-b8c21e644330-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.378759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.378623    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n88gj\" (UniqueName: \"kubernetes.io/projected/9606559d-a2e2-41f0-be90-b8c21e644330-kube-api-access-n88gj\") pod \"payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.379440 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.379414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9606559d-a2e2-41f0-be90-b8c21e644330-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.379440 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.379431    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f3b7026-2584-43c2-b6e4-084c6a40cdab-plugins-config-volume\") pod \"payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") " pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.386807 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.386775    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-524kv\" (UniqueName: \"kubernetes.io/projected/2f3b7026-2584-43c2-b6e4-084c6a40cdab-kube-api-access-524kv\") pod \"payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") " pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.387027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.387009    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n88gj\" (UniqueName: \"kubernetes.io/projected/9606559d-a2e2-41f0-be90-b8c21e644330-kube-api-access-n88gj\") pod \"payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.458941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.458849    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.471258 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.471231    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.612056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.612022    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"]
Jul 28 01:35:58.614261 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:35:58.614229    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9606559d_a2e2_41f0_be90_b8c21e644330.slice/crio-aa6cb1b076bb224d0f8bb53d122446a707f7f8282ccbaa2731bf61bf7fb484f1 WatchSource:0}: Error finding container aa6cb1b076bb224d0f8bb53d122446a707f7f8282ccbaa2731bf61bf7fb484f1: Status 404 returned error can't find the container with id aa6cb1b076bb224d0f8bb53d122446a707f7f8282ccbaa2731bf61bf7fb484f1
Jul 28 01:35:58.639025 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.638995    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"]
Jul 28 01:35:58.655046 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:35:58.655015    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f3b7026_2584_43c2_b6e4_084c6a40cdab.slice/crio-ea877a76bbd93f97ab4ac06ff0ad382b8c2ecc07e24850ef72aa92ec0c68e68d WatchSource:0}: Error finding container ea877a76bbd93f97ab4ac06ff0ad382b8c2ecc07e24850ef72aa92ec0c68e68d: Status 404 returned error can't find the container with id ea877a76bbd93f97ab4ac06ff0ad382b8c2ecc07e24850ef72aa92ec0c68e68d
Jul 28 01:35:58.860703 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.860661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd" event={"ID":"2f3b7026-2584-43c2-b6e4-084c6a40cdab","Type":"ContainerStarted","Data":"40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2"}
Jul 28 01:35:58.860904 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.860737    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:35:58.860904 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.860760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd" event={"ID":"2f3b7026-2584-43c2-b6e4-084c6a40cdab","Type":"ContainerStarted","Data":"ea877a76bbd93f97ab4ac06ff0ad382b8c2ecc07e24850ef72aa92ec0c68e68d"}
Jul 28 01:35:58.869292 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.869259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2" event={"ID":"9606559d-a2e2-41f0-be90-b8c21e644330","Type":"ContainerStarted","Data":"a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e"}
Jul 28 01:35:58.869424 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.869338    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2" event={"ID":"9606559d-a2e2-41f0-be90-b8c21e644330","Type":"ContainerStarted","Data":"aa6cb1b076bb224d0f8bb53d122446a707f7f8282ccbaa2731bf61bf7fb484f1"}
Jul 28 01:35:58.869424 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.869364    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:35:58.879598 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.879554    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd" podStartSLOduration=0.879539455 podStartE2EDuration="879.539455ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:58.87827653 +0000 UTC m=+1894.889515155" watchObservedRunningTime="2026-07-28 01:35:58.879539455 +0000 UTC m=+1894.890778066"
Jul 28 01:35:58.895676 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:35:58.895619    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2" podStartSLOduration=0.895602447 podStartE2EDuration="895.602447ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:35:58.894933124 +0000 UTC m=+1894.906171733" watchObservedRunningTime="2026-07-28 01:35:58.895602447 +0000 UTC m=+1894.906841055"
Jul 28 01:36:06.836705 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:06.836671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:36:09.876653 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:09.876623    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:36:09.877062 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:09.876808    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:36:22.149443 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.149402    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"]
Jul 28 01:36:22.153936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.153918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:22.156996 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.156952    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:36:22.157219 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.157200    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:36:22.160058 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.160038    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"]
Jul 28 01:36:22.180926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.180884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn\" (UID: \"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:22.281958 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.281917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn\" (UID: \"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:22.282576 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.282557    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn\" (UID: \"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:22.465414 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.465325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:22.600343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.600312    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"]
Jul 28 01:36:22.603117 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:36:22.603087    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7150c8_f84d_4d6c_94ce_e2ea0ffb5212.slice/crio-65fd32847efac7d0fa0af350889c2e9bbf98095cb620042a33d007c580bcdc1c WatchSource:0}: Error finding container 65fd32847efac7d0fa0af350889c2e9bbf98095cb620042a33d007c580bcdc1c: Status 404 returned error can't find the container with id 65fd32847efac7d0fa0af350889c2e9bbf98095cb620042a33d007c580bcdc1c
Jul 28 01:36:22.991593 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.991557    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn" event={"ID":"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212","Type":"ContainerStarted","Data":"946bd31d98ed5757882635749c8e9930d61bfb217f7dd19658d3635bfaa107e3"}
Jul 28 01:36:22.991774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:22.991614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn" event={"ID":"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212","Type":"ContainerStarted","Data":"65fd32847efac7d0fa0af350889c2e9bbf98095cb620042a33d007c580bcdc1c"}
Jul 28 01:36:23.007769 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:23.007717    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn" podStartSLOduration=1.007703692 podStartE2EDuration="1.007703692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:36:23.006419758 +0000 UTC m=+1919.017658368" watchObservedRunningTime="2026-07-28 01:36:23.007703692 +0000 UTC m=+1919.018942300"
Jul 28 01:36:23.997604 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:23.997569    2566 generic.go:356] "Generic (PLEG): container finished" podID="8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212" containerID="946bd31d98ed5757882635749c8e9930d61bfb217f7dd19658d3635bfaa107e3" exitCode=0
Jul 28 01:36:23.998015 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:23.997651    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn" event={"ID":"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212","Type":"ContainerDied","Data":"946bd31d98ed5757882635749c8e9930d61bfb217f7dd19658d3635bfaa107e3"}
Jul 28 01:36:25.137102 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:25.137073    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:25.207868 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:25.207831    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212-maas-api-service-ca\") pod \"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212\" (UID: \"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212\") "
Jul 28 01:36:25.208244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:25.208218    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212" (UID: "8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:36:25.309214 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:25.309128    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:36:26.008294 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:26.008260    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"
Jul 28 01:36:26.008294 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:26.008288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn" event={"ID":"8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212","Type":"ContainerDied","Data":"65fd32847efac7d0fa0af350889c2e9bbf98095cb620042a33d007c580bcdc1c"}
Jul 28 01:36:26.008524 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:26.008335    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65fd32847efac7d0fa0af350889c2e9bbf98095cb620042a33d007c580bcdc1c"
Jul 28 01:36:27.916698 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:27.916657    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"]
Jul 28 01:36:27.917213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:27.916910    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd" podUID="2f3b7026-2584-43c2-b6e4-084c6a40cdab" containerName="payload-processing" containerID="cri-o://40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2" gracePeriod=30
Jul 28 01:36:27.934133 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:27.929119    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"]
Jul 28 01:36:27.934133 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:27.929350    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2" podUID="9606559d-a2e2-41f0-be90-b8c21e644330" containerName="payload-pre-processing" containerID="cri-o://a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e" gracePeriod=30
Jul 28 01:36:28.080566 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.080023    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"]
Jul 28 01:36:28.082342 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.082313    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-30dd5a44-hlmkn"]
Jul 28 01:36:28.208013 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.207985    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:36:28.211221 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.211198    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:36:28.237540 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.237510    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9606559d-a2e2-41f0-be90-b8c21e644330-plugins-config-volume\") pod \"9606559d-a2e2-41f0-be90-b8c21e644330\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") "
Jul 28 01:36:28.237732 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.237568    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f3b7026-2584-43c2-b6e4-084c6a40cdab-plugins-config-volume\") pod \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") "
Jul 28 01:36:28.237732 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.237640    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-524kv\" (UniqueName: \"kubernetes.io/projected/2f3b7026-2584-43c2-b6e4-084c6a40cdab-kube-api-access-524kv\") pod \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\" (UID: \"2f3b7026-2584-43c2-b6e4-084c6a40cdab\") "
Jul 28 01:36:28.237732 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.237670    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n88gj\" (UniqueName: \"kubernetes.io/projected/9606559d-a2e2-41f0-be90-b8c21e644330-kube-api-access-n88gj\") pod \"9606559d-a2e2-41f0-be90-b8c21e644330\" (UID: \"9606559d-a2e2-41f0-be90-b8c21e644330\") "
Jul 28 01:36:28.237953 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.237934    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9606559d-a2e2-41f0-be90-b8c21e644330-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9606559d-a2e2-41f0-be90-b8c21e644330" (UID: "9606559d-a2e2-41f0-be90-b8c21e644330"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:36:28.238068 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.237959    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f3b7026-2584-43c2-b6e4-084c6a40cdab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f3b7026-2584-43c2-b6e4-084c6a40cdab" (UID: "2f3b7026-2584-43c2-b6e4-084c6a40cdab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:36:28.239984 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.239947    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f3b7026-2584-43c2-b6e4-084c6a40cdab-kube-api-access-524kv" (OuterVolumeSpecName: "kube-api-access-524kv") pod "2f3b7026-2584-43c2-b6e4-084c6a40cdab" (UID: "2f3b7026-2584-43c2-b6e4-084c6a40cdab"). InnerVolumeSpecName "kube-api-access-524kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:36:28.239984 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.239958    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9606559d-a2e2-41f0-be90-b8c21e644330-kube-api-access-n88gj" (OuterVolumeSpecName: "kube-api-access-n88gj") pod "9606559d-a2e2-41f0-be90-b8c21e644330" (UID: "9606559d-a2e2-41f0-be90-b8c21e644330"). InnerVolumeSpecName "kube-api-access-n88gj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:36:28.339245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.339202    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9606559d-a2e2-41f0-be90-b8c21e644330-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.339245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.339234    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f3b7026-2584-43c2-b6e4-084c6a40cdab-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.339245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.339243    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-524kv\" (UniqueName: \"kubernetes.io/projected/2f3b7026-2584-43c2-b6e4-084c6a40cdab-kube-api-access-524kv\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.339245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.339252    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n88gj\" (UniqueName: \"kubernetes.io/projected/9606559d-a2e2-41f0-be90-b8c21e644330-kube-api-access-n88gj\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:36:28.566055 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:28.561705    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212" path="/var/lib/kubelet/pods/8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212/volumes"
Jul 28 01:36:29.025430 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.025398    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f3b7026-2584-43c2-b6e4-084c6a40cdab" containerID="40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2" exitCode=0
Jul 28 01:36:29.025928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.025470    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"
Jul 28 01:36:29.025928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.025500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd" event={"ID":"2f3b7026-2584-43c2-b6e4-084c6a40cdab","Type":"ContainerDied","Data":"40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2"}
Jul 28 01:36:29.025928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.025562    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd" event={"ID":"2f3b7026-2584-43c2-b6e4-084c6a40cdab","Type":"ContainerDied","Data":"ea877a76bbd93f97ab4ac06ff0ad382b8c2ecc07e24850ef72aa92ec0c68e68d"}
Jul 28 01:36:29.025928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.025597    2566 scope.go:117] "RemoveContainer" containerID="40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2"
Jul 28 01:36:29.027026 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.026924    2566 generic.go:356] "Generic (PLEG): container finished" podID="9606559d-a2e2-41f0-be90-b8c21e644330" containerID="a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e" exitCode=0
Jul 28 01:36:29.027026 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.027015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2" event={"ID":"9606559d-a2e2-41f0-be90-b8c21e644330","Type":"ContainerDied","Data":"a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e"}
Jul 28 01:36:29.027164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.027025    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"
Jul 28 01:36:29.027164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.027049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2" event={"ID":"9606559d-a2e2-41f0-be90-b8c21e644330","Type":"ContainerDied","Data":"aa6cb1b076bb224d0f8bb53d122446a707f7f8282ccbaa2731bf61bf7fb484f1"}
Jul 28 01:36:29.035887 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.035863    2566 scope.go:117] "RemoveContainer" containerID="40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2"
Jul 28 01:36:29.036398 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:36:29.036370    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2\": container with ID starting with 40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2 not found: ID does not exist" containerID="40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2"
Jul 28 01:36:29.036573 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.036408    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2"} err="failed to get container status \"40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2\": rpc error: code = NotFound desc = could not find container \"40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2\": container with ID starting with 40036b42672e74df37a539149a43fccceec9be0b05c5b1c6db516b3913a740e2 not found: ID does not exist"
Jul 28 01:36:29.036573 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.036435    2566 scope.go:117] "RemoveContainer" containerID="a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e"
Jul 28 01:36:29.045489 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.045471    2566 scope.go:117] "RemoveContainer" containerID="a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e"
Jul 28 01:36:29.045781 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:36:29.045759    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e\": container with ID starting with a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e not found: ID does not exist" containerID="a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e"
Jul 28 01:36:29.045835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.045790    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e"} err="failed to get container status \"a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e\": rpc error: code = NotFound desc = could not find container \"a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e\": container with ID starting with a93cb6904bf84f1a87cd0d9e403933ed4ebdc6a29f2bd432d3fcaf38c77b5b1e not found: ID does not exist"
Jul 28 01:36:29.075779 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.075744    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"]
Jul 28 01:36:29.086939 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.086907    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-30dd5a44-b95475576-2rpp2"]
Jul 28 01:36:29.162189 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.162154    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"]
Jul 28 01:36:29.172748 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:29.172712    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-30dd5a44-568dc6455d-cgnsd"]
Jul 28 01:36:30.560747 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:30.560708    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f3b7026-2584-43c2-b6e4-084c6a40cdab" path="/var/lib/kubelet/pods/2f3b7026-2584-43c2-b6e4-084c6a40cdab/volumes"
Jul 28 01:36:30.561185 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:30.561150    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9606559d-a2e2-41f0-be90-b8c21e644330" path="/var/lib/kubelet/pods/9606559d-a2e2-41f0-be90-b8c21e644330/volumes"
Jul 28 01:36:31.733284 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:36:31.733246    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:37:57.837273 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:37:57.837237    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:38:10.739345 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:10.739305    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:38:20.131334 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:20.131293    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:38:26.919730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.919692    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"]
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920071    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9606559d-a2e2-41f0-be90-b8c21e644330" containerName="payload-pre-processing"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920084    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9606559d-a2e2-41f0-be90-b8c21e644330" containerName="payload-pre-processing"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920101    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212" containerName="revoke-keys"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920106    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212" containerName="revoke-keys"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920115    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f3b7026-2584-43c2-b6e4-084c6a40cdab" containerName="payload-processing"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920121    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f3b7026-2584-43c2-b6e4-084c6a40cdab" containerName="payload-processing"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920172    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7150c8-f84d-4d6c-94ce-e2ea0ffb5212" containerName="revoke-keys"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920179    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f3b7026-2584-43c2-b6e4-084c6a40cdab" containerName="payload-processing"
Jul 28 01:38:26.920217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.920186    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9606559d-a2e2-41f0-be90-b8c21e644330" containerName="payload-pre-processing"
Jul 28 01:38:26.923103 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.923085    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:26.925858 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.925834    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6046872f\""
Jul 28 01:38:26.936606 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.936580    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"]
Jul 28 01:38:26.938150 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.938127    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"]
Jul 28 01:38:26.941274 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.941256    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:26.944436 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.944416    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6046872f-dockercfg-kvttb\""
Jul 28 01:38:26.953377 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:26.953354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"]
Jul 28 01:38:27.069158 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.069117    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.069350 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.069181    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwrgl\" (UniqueName: \"kubernetes.io/projected/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-kube-api-access-wwrgl\") pod \"payload-processing-e2e-mt-6046872f-594469df58-cc5dw\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") " pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.069350 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.069239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-plugins-config-volume\") pod \"payload-processing-e2e-mt-6046872f-594469df58-cc5dw\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") " pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.069350 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.069265    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmpp9\" (UniqueName: \"kubernetes.io/projected/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-kube-api-access-wmpp9\") pod \"payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.170728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.170639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.170728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.170694    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrgl\" (UniqueName: \"kubernetes.io/projected/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-kube-api-access-wwrgl\") pod \"payload-processing-e2e-mt-6046872f-594469df58-cc5dw\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") " pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.170935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.170755    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-plugins-config-volume\") pod \"payload-processing-e2e-mt-6046872f-594469df58-cc5dw\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") " pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.170935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.170777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmpp9\" (UniqueName: \"kubernetes.io/projected/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-kube-api-access-wmpp9\") pod \"payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.171469 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.171447    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.171533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.171457    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-plugins-config-volume\") pod \"payload-processing-e2e-mt-6046872f-594469df58-cc5dw\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") " pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.180361 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.180329    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwrgl\" (UniqueName: \"kubernetes.io/projected/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-kube-api-access-wwrgl\") pod \"payload-processing-e2e-mt-6046872f-594469df58-cc5dw\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") " pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.181383 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.181363    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmpp9\" (UniqueName: \"kubernetes.io/projected/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-kube-api-access-wmpp9\") pod \"payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.234473 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.234416    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.253054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.253028    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.398927 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.398898    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"]
Jul 28 01:38:27.402383 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:38:27.402353    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc9c83c_0e89_41ed_a76c_d9b3bdb21ef4.slice/crio-3d6868927aad400062347326d01c8a536e0f003017a04b24de3efa49e74cda94 WatchSource:0}: Error finding container 3d6868927aad400062347326d01c8a536e0f003017a04b24de3efa49e74cda94: Status 404 returned error can't find the container with id 3d6868927aad400062347326d01c8a536e0f003017a04b24de3efa49e74cda94
Jul 28 01:38:27.413560 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.413533    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"]
Jul 28 01:38:27.416531 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:38:27.416497    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14d0379c_a4be_4b34_9ed3_106b2b4bd8c8.slice/crio-f89ea9739da2320f15b79c161e7338ebbc59fd9be1d9f2616d8fa30326e69a23 WatchSource:0}: Error finding container f89ea9739da2320f15b79c161e7338ebbc59fd9be1d9f2616d8fa30326e69a23: Status 404 returned error can't find the container with id f89ea9739da2320f15b79c161e7338ebbc59fd9be1d9f2616d8fa30326e69a23
Jul 28 01:38:27.580884 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.580842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" event={"ID":"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4","Type":"ContainerStarted","Data":"8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0"}
Jul 28 01:38:27.581107 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.580913    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:27.581107 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.580935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" event={"ID":"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4","Type":"ContainerStarted","Data":"3d6868927aad400062347326d01c8a536e0f003017a04b24de3efa49e74cda94"}
Jul 28 01:38:27.582497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.582470    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" event={"ID":"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8","Type":"ContainerStarted","Data":"2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7"}
Jul 28 01:38:27.582656 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.582513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" event={"ID":"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8","Type":"ContainerStarted","Data":"f89ea9739da2320f15b79c161e7338ebbc59fd9be1d9f2616d8fa30326e69a23"}
Jul 28 01:38:27.582656 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.582533    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:27.597605 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.597549    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" podStartSLOduration=1.597530245 podStartE2EDuration="1.597530245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:38:27.59712166 +0000 UTC m=+2043.608360269" watchObservedRunningTime="2026-07-28 01:38:27.597530245 +0000 UTC m=+2043.608768856"
Jul 28 01:38:27.616045 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:27.615995    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" podStartSLOduration=1.615961389 podStartE2EDuration="1.615961389s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:38:27.615151021 +0000 UTC m=+2043.626389630" watchObservedRunningTime="2026-07-28 01:38:27.615961389 +0000 UTC m=+2043.627199998"
Jul 28 01:38:30.141737 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:30.141701    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:38:38.588733 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:38.588699    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:38:38.589225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:38.588767    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:38:57.980577 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:57.980494    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"]
Jul 28 01:38:57.984077 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:57.984060    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:38:57.986626 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:57.986606    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:38:57.986626 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:57.986617    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:38:57.992223 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:57.992196    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"]
Jul 28 01:38:58.033399 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:58.033364    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ece613-0856-4d29-9f87-43b3da379d30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6046872f-bd9tp\" (UID: \"c6ece613-0856-4d29-9f87-43b3da379d30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:38:58.134165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:58.134123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ece613-0856-4d29-9f87-43b3da379d30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6046872f-bd9tp\" (UID: \"c6ece613-0856-4d29-9f87-43b3da379d30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:38:58.134753 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:58.134732    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ece613-0856-4d29-9f87-43b3da379d30-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6046872f-bd9tp\" (UID: \"c6ece613-0856-4d29-9f87-43b3da379d30\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:38:58.296756 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:58.296667    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:38:58.437228 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:58.437202    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"]
Jul 28 01:38:58.439775 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:38:58.439742    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ece613_0856_4d29_9f87_43b3da379d30.slice/crio-70b6db5e109e3cf7e228c98c0f8eb5f930aa2ebc5712d22f8c748cdd99a37b8f WatchSource:0}: Error finding container 70b6db5e109e3cf7e228c98c0f8eb5f930aa2ebc5712d22f8c748cdd99a37b8f: Status 404 returned error can't find the container with id 70b6db5e109e3cf7e228c98c0f8eb5f930aa2ebc5712d22f8c748cdd99a37b8f
Jul 28 01:38:58.745701 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:58.745667    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp" event={"ID":"c6ece613-0856-4d29-9f87-43b3da379d30","Type":"ContainerStarted","Data":"70b6db5e109e3cf7e228c98c0f8eb5f930aa2ebc5712d22f8c748cdd99a37b8f"}
Jul 28 01:38:59.751600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:59.751564    2566 generic.go:356] "Generic (PLEG): container finished" podID="c6ece613-0856-4d29-9f87-43b3da379d30" containerID="8491adb227ac20828800e4aacbb0d923378b06d9093c471102dc28a39bb17215" exitCode=0
Jul 28 01:38:59.752027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:38:59.751619    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp" event={"ID":"c6ece613-0856-4d29-9f87-43b3da379d30","Type":"ContainerDied","Data":"8491adb227ac20828800e4aacbb0d923378b06d9093c471102dc28a39bb17215"}
Jul 28 01:39:00.887629 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:00.887598    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:39:00.955217 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:00.955180    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ece613-0856-4d29-9f87-43b3da379d30-maas-api-service-ca\") pod \"c6ece613-0856-4d29-9f87-43b3da379d30\" (UID: \"c6ece613-0856-4d29-9f87-43b3da379d30\") "
Jul 28 01:39:00.955579 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:00.955553    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6ece613-0856-4d29-9f87-43b3da379d30-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c6ece613-0856-4d29-9f87-43b3da379d30" (UID: "c6ece613-0856-4d29-9f87-43b3da379d30"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:39:01.056616 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:01.056528    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6ece613-0856-4d29-9f87-43b3da379d30-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:39:01.762590 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:01.762562    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"
Jul 28 01:39:01.762763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:01.762561    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp" event={"ID":"c6ece613-0856-4d29-9f87-43b3da379d30","Type":"ContainerDied","Data":"70b6db5e109e3cf7e228c98c0f8eb5f930aa2ebc5712d22f8c748cdd99a37b8f"}
Jul 28 01:39:01.762763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:01.762685    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70b6db5e109e3cf7e228c98c0f8eb5f930aa2ebc5712d22f8c748cdd99a37b8f"
Jul 28 01:39:08.452619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.452585    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"]
Jul 28 01:39:08.453161 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.452822    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" podUID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" containerName="payload-processing" containerID="cri-o://2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7" gracePeriod=30
Jul 28 01:39:08.463456 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.463426    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"]
Jul 28 01:39:08.463695 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.463671    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" podUID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" containerName="payload-pre-processing" containerID="cri-o://8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0" gracePeriod=30
Jul 28 01:39:08.587671 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.587634    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" podUID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" containerName="payload-processing" probeResult="failure" output="dial tcp 10.133.0.46:9004: connect: connection refused"
Jul 28 01:39:08.587851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.587724    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" podUID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.45:9004: connect: connection refused"
Jul 28 01:39:08.600808 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.600727    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"]
Jul 28 01:39:08.603421 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.603387    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6046872f-bd9tp"]
Jul 28 01:39:08.735415 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.735239    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:39:08.738895 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.738856    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:39:08.798341 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.798311    2566 generic.go:356] "Generic (PLEG): container finished" podID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" containerID="8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0" exitCode=0
Jul 28 01:39:08.798529 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.798355    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"
Jul 28 01:39:08.798529 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.798399    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" event={"ID":"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4","Type":"ContainerDied","Data":"8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0"}
Jul 28 01:39:08.798529 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.798454    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg" event={"ID":"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4","Type":"ContainerDied","Data":"3d6868927aad400062347326d01c8a536e0f003017a04b24de3efa49e74cda94"}
Jul 28 01:39:08.798707 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.798533    2566 scope.go:117] "RemoveContainer" containerID="8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0"
Jul 28 01:39:08.799690 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.799671    2566 generic.go:356] "Generic (PLEG): container finished" podID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" containerID="2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7" exitCode=0
Jul 28 01:39:08.799763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.799720    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"
Jul 28 01:39:08.799820 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.799721    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" event={"ID":"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8","Type":"ContainerDied","Data":"2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7"}
Jul 28 01:39:08.799862 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.799851    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw" event={"ID":"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8","Type":"ContainerDied","Data":"f89ea9739da2320f15b79c161e7338ebbc59fd9be1d9f2616d8fa30326e69a23"}
Jul 28 01:39:08.807195 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.807175    2566 scope.go:117] "RemoveContainer" containerID="8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0"
Jul 28 01:39:08.807465 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:39:08.807445    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0\": container with ID starting with 8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0 not found: ID does not exist" containerID="8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0"
Jul 28 01:39:08.807539 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.807479    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0"} err="failed to get container status \"8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0\": rpc error: code = NotFound desc = could not find container \"8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0\": container with ID starting with 8a3810aa418f4795cf5d8fafb6be5713563db34ad8ebc29f81f1395c7c4cdfa0 not found: ID does not exist"
Jul 28 01:39:08.807539 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.807500    2566 scope.go:117] "RemoveContainer" containerID="2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7"
Jul 28 01:39:08.815376 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.815360    2566 scope.go:117] "RemoveContainer" containerID="2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7"
Jul 28 01:39:08.815616 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:39:08.815600    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7\": container with ID starting with 2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7 not found: ID does not exist" containerID="2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7"
Jul 28 01:39:08.815659 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.815624    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7"} err="failed to get container status \"2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7\": rpc error: code = NotFound desc = could not find container \"2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7\": container with ID starting with 2ce43e3af639825b710c3ff28e286da8429a8da385532dc258e783b5c385d8a7 not found: ID does not exist"
Jul 28 01:39:08.819953 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.819934    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmpp9\" (UniqueName: \"kubernetes.io/projected/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-kube-api-access-wmpp9\") pod \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") "
Jul 28 01:39:08.820062 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.819987    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-plugins-config-volume\") pod \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\" (UID: \"5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4\") "
Jul 28 01:39:08.820062 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.820013    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwrgl\" (UniqueName: \"kubernetes.io/projected/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-kube-api-access-wwrgl\") pod \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") "
Jul 28 01:39:08.820062 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.820039    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-plugins-config-volume\") pod \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\" (UID: \"14d0379c-a4be-4b34-9ed3-106b2b4bd8c8\") "
Jul 28 01:39:08.820444 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.820418    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" (UID: "5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:39:08.820542 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.820465    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" (UID: "14d0379c-a4be-4b34-9ed3-106b2b4bd8c8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:39:08.822262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.822242    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-kube-api-access-wmpp9" (OuterVolumeSpecName: "kube-api-access-wmpp9") pod "5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" (UID: "5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4"). InnerVolumeSpecName "kube-api-access-wmpp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:39:08.822324 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.822273    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-kube-api-access-wwrgl" (OuterVolumeSpecName: "kube-api-access-wwrgl") pod "14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" (UID: "14d0379c-a4be-4b34-9ed3-106b2b4bd8c8"). InnerVolumeSpecName "kube-api-access-wwrgl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:39:08.921108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.921073    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:39:08.921108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.921101    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwrgl\" (UniqueName: \"kubernetes.io/projected/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-kube-api-access-wwrgl\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:39:08.921108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.921111    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:39:08.921108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:08.921119    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmpp9\" (UniqueName: \"kubernetes.io/projected/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4-kube-api-access-wmpp9\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:39:09.123002 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:09.122954    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"]
Jul 28 01:39:09.126516 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:09.126486    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6046872f-ddf5954c6-tjvxg"]
Jul 28 01:39:09.137854 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:09.137824    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"]
Jul 28 01:39:09.141309 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:09.141290    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6046872f-594469df58-cc5dw"]
Jul 28 01:39:10.560854 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:10.560810    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" path="/var/lib/kubelet/pods/14d0379c-a4be-4b34-9ed3-106b2b4bd8c8/volumes"
Jul 28 01:39:10.561371 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:10.561345    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" path="/var/lib/kubelet/pods/5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4/volumes"
Jul 28 01:39:10.561689 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:10.561670    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6ece613-0856-4d29-9f87-43b3da379d30" path="/var/lib/kubelet/pods/c6ece613-0856-4d29-9f87-43b3da379d30/volumes"
Jul 28 01:39:11.229749 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:11.229711    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:39:39.024349 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024315    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"]
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024637    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" containerName="payload-processing"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024648    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" containerName="payload-processing"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024662    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" containerName="payload-pre-processing"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024668    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" containerName="payload-pre-processing"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024679    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6ece613-0856-4d29-9f87-43b3da379d30" containerName="revoke-keys"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024685    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6ece613-0856-4d29-9f87-43b3da379d30" containerName="revoke-keys"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024744    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="14d0379c-a4be-4b34-9ed3-106b2b4bd8c8" containerName="payload-processing"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024753    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6ece613-0856-4d29-9f87-43b3da379d30" containerName="revoke-keys"
Jul 28 01:39:39.024835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.024761    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fc9c83c-0e89-41ed-a76c-d9b3bdb21ef4" containerName="payload-pre-processing"
Jul 28 01:39:39.027842 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.027808    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.028512 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.028488    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"]
Jul 28 01:39:39.030442 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.030423    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f0341a4c\""
Jul 28 01:39:39.030563 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.030423    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f0341a4c-dockercfg-zlbjq\""
Jul 28 01:39:39.031616 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.031566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.038152 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.038131    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"]
Jul 28 01:39:39.047747 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.047692    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"]
Jul 28 01:39:39.184640 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.184605    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkqtr\" (UniqueName: \"kubernetes.io/projected/b5412841-790b-49fe-b9a3-9d5a3a448339-kube-api-access-fkqtr\") pod \"payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.184640 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.184643    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw4bj\" (UniqueName: \"kubernetes.io/projected/3b99528d-7a02-439c-923c-59b5f3206ba9-kube-api-access-nw4bj\") pod \"payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") " pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.184875 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.184676    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5412841-790b-49fe-b9a3-9d5a3a448339-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.184875 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.184692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b99528d-7a02-439c-923c-59b5f3206ba9-plugins-config-volume\") pod \"payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") " pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.286222 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.286124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkqtr\" (UniqueName: \"kubernetes.io/projected/b5412841-790b-49fe-b9a3-9d5a3a448339-kube-api-access-fkqtr\") pod \"payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.286222 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.286169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw4bj\" (UniqueName: \"kubernetes.io/projected/3b99528d-7a02-439c-923c-59b5f3206ba9-kube-api-access-nw4bj\") pod \"payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") " pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.286222 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.286213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5412841-790b-49fe-b9a3-9d5a3a448339-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.286511 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.286232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b99528d-7a02-439c-923c-59b5f3206ba9-plugins-config-volume\") pod \"payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") " pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.287164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.287137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5412841-790b-49fe-b9a3-9d5a3a448339-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.287271 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.287137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b99528d-7a02-439c-923c-59b5f3206ba9-plugins-config-volume\") pod \"payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") " pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.297167 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.297138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw4bj\" (UniqueName: \"kubernetes.io/projected/3b99528d-7a02-439c-923c-59b5f3206ba9-kube-api-access-nw4bj\") pod \"payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") " pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.297274 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.297230    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkqtr\" (UniqueName: \"kubernetes.io/projected/b5412841-790b-49fe-b9a3-9d5a3a448339-kube-api-access-fkqtr\") pod \"payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.340385 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.340348    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.347647 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.347606    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.489538 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.489506    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"]
Jul 28 01:39:39.491881 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:39:39.491848    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5412841_790b_49fe_b9a3_9d5a3a448339.slice/crio-453dccb675ef16c38f0fb18bb5af3b2b9233d37e851d811cdf37b780ed21dba8 WatchSource:0}: Error finding container 453dccb675ef16c38f0fb18bb5af3b2b9233d37e851d811cdf37b780ed21dba8: Status 404 returned error can't find the container with id 453dccb675ef16c38f0fb18bb5af3b2b9233d37e851d811cdf37b780ed21dba8
Jul 28 01:39:39.514289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.514243    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"]
Jul 28 01:39:39.516385 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:39:39.516358    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b99528d_7a02_439c_923c_59b5f3206ba9.slice/crio-d684d5b5a06d22bb3d1080fd891ccfe6c7a2de3280568ec87eaa0e63a3d48584 WatchSource:0}: Error finding container d684d5b5a06d22bb3d1080fd891ccfe6c7a2de3280568ec87eaa0e63a3d48584: Status 404 returned error can't find the container with id d684d5b5a06d22bb3d1080fd891ccfe6c7a2de3280568ec87eaa0e63a3d48584
Jul 28 01:39:39.954891 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.954850    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85" event={"ID":"3b99528d-7a02-439c-923c-59b5f3206ba9","Type":"ContainerStarted","Data":"643241a1d6e2e536f194060985fc1faa540855445d2718e8c15bb9d7f9d1f15e"}
Jul 28 01:39:39.955121 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.954910    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85" event={"ID":"3b99528d-7a02-439c-923c-59b5f3206ba9","Type":"ContainerStarted","Data":"d684d5b5a06d22bb3d1080fd891ccfe6c7a2de3280568ec87eaa0e63a3d48584"}
Jul 28 01:39:39.955121 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.954957    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:39:39.956778 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.956751    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b" event={"ID":"b5412841-790b-49fe-b9a3-9d5a3a448339","Type":"ContainerStarted","Data":"ce13f6afd4f0bbe664ca21acdd12c9818ebc95b4ddfc4b4a28ad7fa75860d439"}
Jul 28 01:39:39.956894 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.956799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b" event={"ID":"b5412841-790b-49fe-b9a3-9d5a3a448339","Type":"ContainerStarted","Data":"453dccb675ef16c38f0fb18bb5af3b2b9233d37e851d811cdf37b780ed21dba8"}
Jul 28 01:39:39.956894 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.956827    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:39.974083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.974023    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85" podStartSLOduration=0.974006924 podStartE2EDuration="974.006924ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:39:39.972096707 +0000 UTC m=+2115.983335321" watchObservedRunningTime="2026-07-28 01:39:39.974006924 +0000 UTC m=+2115.985245534"
Jul 28 01:39:39.986879 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:39.986816    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b" podStartSLOduration=0.986794722 podStartE2EDuration="986.794722ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:39:39.98654198 +0000 UTC m=+2115.997780603" watchObservedRunningTime="2026-07-28 01:39:39.986794722 +0000 UTC m=+2115.998033333"
Jul 28 01:39:45.036449 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:45.036410    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:39:50.964799 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:50.964767    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:39:50.965246 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:39:50.964835    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:40:11.333196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:11.333158    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:40:28.132807 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:28.132771    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:40:32.533330 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.533245    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"]
Jul 28 01:40:32.536618 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.536601    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:32.539262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.539235    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:40:32.539382 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.539242    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:40:32.544284 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.544226    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"]
Jul 28 01:40:32.643038 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.642999    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e782f05b-015f-4967-a3a8-cb9e198a61db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz\" (UID: \"e782f05b-015f-4967-a3a8-cb9e198a61db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:32.744164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.744124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e782f05b-015f-4967-a3a8-cb9e198a61db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz\" (UID: \"e782f05b-015f-4967-a3a8-cb9e198a61db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:32.744792 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.744770    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e782f05b-015f-4967-a3a8-cb9e198a61db-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz\" (UID: \"e782f05b-015f-4967-a3a8-cb9e198a61db\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:32.848538 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.848505    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:32.982704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:32.982681    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"]
Jul 28 01:40:32.984950 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:40:32.984906    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode782f05b_015f_4967_a3a8_cb9e198a61db.slice/crio-c5710b5bcbc5f23d32b8a722b0682091d6809a8d0f210410ff955ff7f777e7d9 WatchSource:0}: Error finding container c5710b5bcbc5f23d32b8a722b0682091d6809a8d0f210410ff955ff7f777e7d9: Status 404 returned error can't find the container with id c5710b5bcbc5f23d32b8a722b0682091d6809a8d0f210410ff955ff7f777e7d9
Jul 28 01:40:33.212866 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:33.212827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz" event={"ID":"e782f05b-015f-4967-a3a8-cb9e198a61db","Type":"ContainerStarted","Data":"cdc6a9d15e4724cec01ad6d60c31de4b5c1350c5175ed9b100cb6bcf0f6aa68a"}
Jul 28 01:40:33.213054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:33.212893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz" event={"ID":"e782f05b-015f-4967-a3a8-cb9e198a61db","Type":"ContainerStarted","Data":"c5710b5bcbc5f23d32b8a722b0682091d6809a8d0f210410ff955ff7f777e7d9"}
Jul 28 01:40:33.230480 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:33.230428    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz" podStartSLOduration=1.230410736 podStartE2EDuration="1.230410736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:40:33.229005503 +0000 UTC m=+2169.240244114" watchObservedRunningTime="2026-07-28 01:40:33.230410736 +0000 UTC m=+2169.241649552"
Jul 28 01:40:34.218981 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:34.218940    2566 generic.go:356] "Generic (PLEG): container finished" podID="e782f05b-015f-4967-a3a8-cb9e198a61db" containerID="cdc6a9d15e4724cec01ad6d60c31de4b5c1350c5175ed9b100cb6bcf0f6aa68a" exitCode=0
Jul 28 01:40:34.219404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:34.219024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz" event={"ID":"e782f05b-015f-4967-a3a8-cb9e198a61db","Type":"ContainerDied","Data":"cdc6a9d15e4724cec01ad6d60c31de4b5c1350c5175ed9b100cb6bcf0f6aa68a"}
Jul 28 01:40:35.358708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:35.358683    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:35.466106 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:35.466077    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e782f05b-015f-4967-a3a8-cb9e198a61db-maas-api-service-ca\") pod \"e782f05b-015f-4967-a3a8-cb9e198a61db\" (UID: \"e782f05b-015f-4967-a3a8-cb9e198a61db\") "
Jul 28 01:40:35.466432 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:35.466407    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e782f05b-015f-4967-a3a8-cb9e198a61db-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e782f05b-015f-4967-a3a8-cb9e198a61db" (UID: "e782f05b-015f-4967-a3a8-cb9e198a61db"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:35.566900 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:35.566817    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e782f05b-015f-4967-a3a8-cb9e198a61db-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:40:36.231151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:36.231124    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"
Jul 28 01:40:36.231343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:36.231115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz" event={"ID":"e782f05b-015f-4967-a3a8-cb9e198a61db","Type":"ContainerDied","Data":"c5710b5bcbc5f23d32b8a722b0682091d6809a8d0f210410ff955ff7f777e7d9"}
Jul 28 01:40:36.231343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:36.231290    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5710b5bcbc5f23d32b8a722b0682091d6809a8d0f210410ff955ff7f777e7d9"
Jul 28 01:40:46.737189 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:46.737150    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:40:48.097856 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.097807    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"]
Jul 28 01:40:48.098704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.098638    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85" podUID="3b99528d-7a02-439c-923c-59b5f3206ba9" containerName="payload-processing" containerID="cri-o://643241a1d6e2e536f194060985fc1faa540855445d2718e8c15bb9d7f9d1f15e" gracePeriod=30
Jul 28 01:40:48.106899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.106877    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"]
Jul 28 01:40:48.107182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.107159    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b" podUID="b5412841-790b-49fe-b9a3-9d5a3a448339" containerName="payload-pre-processing" containerID="cri-o://ce13f6afd4f0bbe664ca21acdd12c9818ebc95b4ddfc4b4a28ad7fa75860d439" gracePeriod=30
Jul 28 01:40:48.257076 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.257016    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"]
Jul 28 01:40:48.258519 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.258490    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f0341a4c-pkdtz"]
Jul 28 01:40:48.299960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.298173    2566 generic.go:356] "Generic (PLEG): container finished" podID="3b99528d-7a02-439c-923c-59b5f3206ba9" containerID="643241a1d6e2e536f194060985fc1faa540855445d2718e8c15bb9d7f9d1f15e" exitCode=0
Jul 28 01:40:48.299960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.298346    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85" event={"ID":"3b99528d-7a02-439c-923c-59b5f3206ba9","Type":"ContainerDied","Data":"643241a1d6e2e536f194060985fc1faa540855445d2718e8c15bb9d7f9d1f15e"}
Jul 28 01:40:48.301743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.301713    2566 generic.go:356] "Generic (PLEG): container finished" podID="b5412841-790b-49fe-b9a3-9d5a3a448339" containerID="ce13f6afd4f0bbe664ca21acdd12c9818ebc95b4ddfc4b4a28ad7fa75860d439" exitCode=0
Jul 28 01:40:48.301899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.301772    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b" event={"ID":"b5412841-790b-49fe-b9a3-9d5a3a448339","Type":"ContainerDied","Data":"ce13f6afd4f0bbe664ca21acdd12c9818ebc95b4ddfc4b4a28ad7fa75860d439"}
Jul 28 01:40:48.356129 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.356068    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:40:48.387315 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.387287    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:40:48.479676 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.479645    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkqtr\" (UniqueName: \"kubernetes.io/projected/b5412841-790b-49fe-b9a3-9d5a3a448339-kube-api-access-fkqtr\") pod \"b5412841-790b-49fe-b9a3-9d5a3a448339\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") "
Jul 28 01:40:48.479676 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.479678    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b99528d-7a02-439c-923c-59b5f3206ba9-plugins-config-volume\") pod \"3b99528d-7a02-439c-923c-59b5f3206ba9\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") "
Jul 28 01:40:48.479999 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.479706    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw4bj\" (UniqueName: \"kubernetes.io/projected/3b99528d-7a02-439c-923c-59b5f3206ba9-kube-api-access-nw4bj\") pod \"3b99528d-7a02-439c-923c-59b5f3206ba9\" (UID: \"3b99528d-7a02-439c-923c-59b5f3206ba9\") "
Jul 28 01:40:48.479999 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.479766    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5412841-790b-49fe-b9a3-9d5a3a448339-plugins-config-volume\") pod \"b5412841-790b-49fe-b9a3-9d5a3a448339\" (UID: \"b5412841-790b-49fe-b9a3-9d5a3a448339\") "
Jul 28 01:40:48.480210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.480182    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5412841-790b-49fe-b9a3-9d5a3a448339-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5412841-790b-49fe-b9a3-9d5a3a448339" (UID: "b5412841-790b-49fe-b9a3-9d5a3a448339"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:48.480210 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.480192    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b99528d-7a02-439c-923c-59b5f3206ba9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b99528d-7a02-439c-923c-59b5f3206ba9" (UID: "3b99528d-7a02-439c-923c-59b5f3206ba9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:40:48.482130 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.482101    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b99528d-7a02-439c-923c-59b5f3206ba9-kube-api-access-nw4bj" (OuterVolumeSpecName: "kube-api-access-nw4bj") pod "3b99528d-7a02-439c-923c-59b5f3206ba9" (UID: "3b99528d-7a02-439c-923c-59b5f3206ba9"). InnerVolumeSpecName "kube-api-access-nw4bj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:40:48.482209 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.482154    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5412841-790b-49fe-b9a3-9d5a3a448339-kube-api-access-fkqtr" (OuterVolumeSpecName: "kube-api-access-fkqtr") pod "b5412841-790b-49fe-b9a3-9d5a3a448339" (UID: "b5412841-790b-49fe-b9a3-9d5a3a448339"). InnerVolumeSpecName "kube-api-access-fkqtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:40:48.560579 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.560544    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e782f05b-015f-4967-a3a8-cb9e198a61db" path="/var/lib/kubelet/pods/e782f05b-015f-4967-a3a8-cb9e198a61db/volumes"
Jul 28 01:40:48.581317 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.581293    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkqtr\" (UniqueName: \"kubernetes.io/projected/b5412841-790b-49fe-b9a3-9d5a3a448339-kube-api-access-fkqtr\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:40:48.581317 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.581317    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b99528d-7a02-439c-923c-59b5f3206ba9-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:40:48.581459 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.581326    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw4bj\" (UniqueName: \"kubernetes.io/projected/3b99528d-7a02-439c-923c-59b5f3206ba9-kube-api-access-nw4bj\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:40:48.581459 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:48.581335    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5412841-790b-49fe-b9a3-9d5a3a448339-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:40:49.307835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.307803    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"
Jul 28 01:40:49.307835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.307820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85" event={"ID":"3b99528d-7a02-439c-923c-59b5f3206ba9","Type":"ContainerDied","Data":"d684d5b5a06d22bb3d1080fd891ccfe6c7a2de3280568ec87eaa0e63a3d48584"}
Jul 28 01:40:49.308451 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.307895    2566 scope.go:117] "RemoveContainer" containerID="643241a1d6e2e536f194060985fc1faa540855445d2718e8c15bb9d7f9d1f15e"
Jul 28 01:40:49.309262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.309238    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b" event={"ID":"b5412841-790b-49fe-b9a3-9d5a3a448339","Type":"ContainerDied","Data":"453dccb675ef16c38f0fb18bb5af3b2b9233d37e851d811cdf37b780ed21dba8"}
Jul 28 01:40:49.309262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.309252    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"
Jul 28 01:40:49.316734 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.316715    2566 scope.go:117] "RemoveContainer" containerID="ce13f6afd4f0bbe664ca21acdd12c9818ebc95b4ddfc4b4a28ad7fa75860d439"
Jul 28 01:40:49.327211 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.327158    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"]
Jul 28 01:40:49.329325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.329302    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f0341a4c-74d9f96b9d-6hc85"]
Jul 28 01:40:49.339270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.339249    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"]
Jul 28 01:40:49.343733 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:49.343712    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f0341a4c-b5f9466b8-4h74b"]
Jul 28 01:40:50.560311 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:50.560281    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b99528d-7a02-439c-923c-59b5f3206ba9" path="/var/lib/kubelet/pods/3b99528d-7a02-439c-923c-59b5f3206ba9/volumes"
Jul 28 01:40:50.560701 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:50.560639    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5412841-790b-49fe-b9a3-9d5a3a448339" path="/var/lib/kubelet/pods/b5412841-790b-49fe-b9a3-9d5a3a448339/volumes"
Jul 28 01:40:55.529037 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:40:55.528996    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:41:24.678723 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:41:24.678690    2566 scope.go:117] "RemoveContainer" containerID="28ec5de47e78b9503e2a7e12a96e7a1102b7f8ffe48cc309679c390fc093d6e5"
Jul 28 01:41:24.687298 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:41:24.687278    2566 scope.go:117] "RemoveContainer" containerID="48f3bb961fb82cbb4ade9001cac0c006c1000bb1823fd17ebedcd5a13e3d5582"
Jul 28 01:41:43.656781 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:41:43.656746    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:41:55.454222 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:41:55.454127    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:42:24.715562 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:24.715531    2566 scope.go:117] "RemoveContainer" containerID="f8412ed4ceaecd279a87dbf76b73676879cd3c42c60c799a6bbc48c83bace06e"
Jul 28 01:42:24.723711 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:24.723685    2566 scope.go:117] "RemoveContainer" containerID="946bd31d98ed5757882635749c8e9930d61bfb217f7dd19658d3635bfaa107e3"
Jul 28 01:42:39.184450 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184407    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"]
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184737    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e782f05b-015f-4967-a3a8-cb9e198a61db" containerName="revoke-keys"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184750    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e782f05b-015f-4967-a3a8-cb9e198a61db" containerName="revoke-keys"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184759    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b99528d-7a02-439c-923c-59b5f3206ba9" containerName="payload-processing"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184766    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b99528d-7a02-439c-923c-59b5f3206ba9" containerName="payload-processing"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184773    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5412841-790b-49fe-b9a3-9d5a3a448339" containerName="payload-pre-processing"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184779    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5412841-790b-49fe-b9a3-9d5a3a448339" containerName="payload-pre-processing"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184834    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b99528d-7a02-439c-923c-59b5f3206ba9" containerName="payload-processing"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184843    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e782f05b-015f-4967-a3a8-cb9e198a61db" containerName="revoke-keys"
Jul 28 01:42:39.184888 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.184850    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5412841-790b-49fe-b9a3-9d5a3a448339" containerName="payload-pre-processing"
Jul 28 01:42:39.187850 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.187834    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.190370 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.190345    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-45a9e6-dockercfg-77f75\""
Jul 28 01:42:39.190497 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.190463    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-45a9e6\""
Jul 28 01:42:39.199462 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.199439    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"]
Jul 28 01:42:39.251345 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.251304    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"]
Jul 28 01:42:39.254679 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.254661    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.268570 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.268540    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"]
Jul 28 01:42:39.290384 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.290348    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8sm4\" (UniqueName: \"kubernetes.io/projected/dc53eda2-210c-4261-9c7b-b3a192597dc1-kube-api-access-x8sm4\") pod \"payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.290575 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.290505    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc53eda2-210c-4261-9c7b-b3a192597dc1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.390993 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.390936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8sm4\" (UniqueName: \"kubernetes.io/projected/dc53eda2-210c-4261-9c7b-b3a192597dc1-kube-api-access-x8sm4\") pod \"payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.391193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.391011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fef74a6a-182c-478e-a6ac-2e186c02227c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.391193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.391047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7q9l\" (UniqueName: \"kubernetes.io/projected/fef74a6a-182c-478e-a6ac-2e186c02227c-kube-api-access-s7q9l\") pod \"payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.391193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.391075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc53eda2-210c-4261-9c7b-b3a192597dc1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.391780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.391755    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc53eda2-210c-4261-9c7b-b3a192597dc1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.399621 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.399592    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8sm4\" (UniqueName: \"kubernetes.io/projected/dc53eda2-210c-4261-9c7b-b3a192597dc1-kube-api-access-x8sm4\") pod \"payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.492108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.492001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fef74a6a-182c-478e-a6ac-2e186c02227c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.492108 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.492086    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7q9l\" (UniqueName: \"kubernetes.io/projected/fef74a6a-182c-478e-a6ac-2e186c02227c-kube-api-access-s7q9l\") pod \"payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.492705 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.492683    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fef74a6a-182c-478e-a6ac-2e186c02227c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.498814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.498789    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.500687 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.500665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7q9l\" (UniqueName: \"kubernetes.io/projected/fef74a6a-182c-478e-a6ac-2e186c02227c-kube-api-access-s7q9l\") pod \"payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.564330 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.564288    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.644868 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.644838    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"]
Jul 28 01:42:39.646663 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:42:39.646630    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc53eda2_210c_4261_9c7b_b3a192597dc1.slice/crio-1b5413229389f126e136bc6120b851c843a3f923aac28a3de86f0f43675dd901 WatchSource:0}: Error finding container 1b5413229389f126e136bc6120b851c843a3f923aac28a3de86f0f43675dd901: Status 404 returned error can't find the container with id 1b5413229389f126e136bc6120b851c843a3f923aac28a3de86f0f43675dd901
Jul 28 01:42:39.717894 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.717751    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"]
Jul 28 01:42:39.720470 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:42:39.720434    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef74a6a_182c_478e_a6ac_2e186c02227c.slice/crio-314864292da002a7fd9b49f0086cc2680ecbc9501bd66afc7d7c24990667940e WatchSource:0}: Error finding container 314864292da002a7fd9b49f0086cc2680ecbc9501bd66afc7d7c24990667940e: Status 404 returned error can't find the container with id 314864292da002a7fd9b49f0086cc2680ecbc9501bd66afc7d7c24990667940e
Jul 28 01:42:39.836643 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.836604    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9" event={"ID":"dc53eda2-210c-4261-9c7b-b3a192597dc1","Type":"ContainerStarted","Data":"920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f"}
Jul 28 01:42:39.836837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.836657    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9" event={"ID":"dc53eda2-210c-4261-9c7b-b3a192597dc1","Type":"ContainerStarted","Data":"1b5413229389f126e136bc6120b851c843a3f923aac28a3de86f0f43675dd901"}
Jul 28 01:42:39.836837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.836681    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:42:39.838266 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.838238    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp" event={"ID":"fef74a6a-182c-478e-a6ac-2e186c02227c","Type":"ContainerStarted","Data":"1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7"}
Jul 28 01:42:39.838391 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.838288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp" event={"ID":"fef74a6a-182c-478e-a6ac-2e186c02227c","Type":"ContainerStarted","Data":"314864292da002a7fd9b49f0086cc2680ecbc9501bd66afc7d7c24990667940e"}
Jul 28 01:42:39.838391 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.838312    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:39.854908 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.854827    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9" podStartSLOduration=0.854767974 podStartE2EDuration="854.767974ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:42:39.852222326 +0000 UTC m=+2295.863460939" watchObservedRunningTime="2026-07-28 01:42:39.854767974 +0000 UTC m=+2295.866006584"
Jul 28 01:42:39.868504 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:39.868453    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp" podStartSLOduration=0.868431296 podStartE2EDuration="868.431296ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:42:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:42:39.866694367 +0000 UTC m=+2295.877932977" watchObservedRunningTime="2026-07-28 01:42:39.868431296 +0000 UTC m=+2295.879669905"
Jul 28 01:42:43.130407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:43.130370    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:42:50.845075 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:50.845044    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:42:50.845480 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:42:50.845117    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:43:07.399858 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.399819    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"]
Jul 28 01:43:07.403292 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.403275    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.406765 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.406740    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a5bff2-dockercfg-vfwpt\""
Jul 28 01:43:07.407379 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.407353    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a5bff2\""
Jul 28 01:43:07.416933 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.416906    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"]
Jul 28 01:43:07.435505 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.435471    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fde0d0e7-a727-44be-941c-8de65af76918-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.435726 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.435537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkn6q\" (UniqueName: \"kubernetes.io/projected/fde0d0e7-a727-44be-941c-8de65af76918-kube-api-access-bkn6q\") pod \"payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.438571 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.438545    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"]
Jul 28 01:43:07.441935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.441919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.453894 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.453869    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"]
Jul 28 01:43:07.536136 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.536100    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkn6q\" (UniqueName: \"kubernetes.io/projected/fde0d0e7-a727-44be-941c-8de65af76918-kube-api-access-bkn6q\") pod \"payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.536325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.536189    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ed1f71-d988-416a-9ee2-cc494e9497f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.536325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.536226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fde0d0e7-a727-44be-941c-8de65af76918-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.536325 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.536275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7xzr\" (UniqueName: \"kubernetes.io/projected/07ed1f71-d988-416a-9ee2-cc494e9497f5-kube-api-access-k7xzr\") pod \"payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.537024 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.537004    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fde0d0e7-a727-44be-941c-8de65af76918-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.547399 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.547375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkn6q\" (UniqueName: \"kubernetes.io/projected/fde0d0e7-a727-44be-941c-8de65af76918-kube-api-access-bkn6q\") pod \"payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.636851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.636817    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ed1f71-d988-416a-9ee2-cc494e9497f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.637099 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.636892    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7xzr\" (UniqueName: \"kubernetes.io/projected/07ed1f71-d988-416a-9ee2-cc494e9497f5-kube-api-access-k7xzr\") pod \"payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.637706 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.637678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ed1f71-d988-416a-9ee2-cc494e9497f5-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.646556 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.646526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7xzr\" (UniqueName: \"kubernetes.io/projected/07ed1f71-d988-416a-9ee2-cc494e9497f5-kube-api-access-k7xzr\") pod \"payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.714484 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.714386    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.752663 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.752616    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:07.866652 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.866619    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"]
Jul 28 01:43:07.868614 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:43:07.868584    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde0d0e7_a727_44be_941c_8de65af76918.slice/crio-63dfdec809c228398455088f9edd87f17e85e6fae579a64f3396c17fd6e38b90 WatchSource:0}: Error finding container 63dfdec809c228398455088f9edd87f17e85e6fae579a64f3396c17fd6e38b90: Status 404 returned error can't find the container with id 63dfdec809c228398455088f9edd87f17e85e6fae579a64f3396c17fd6e38b90
Jul 28 01:43:07.907568 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.907416    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"]
Jul 28 01:43:07.909489 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:43:07.909464    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ed1f71_d988_416a_9ee2_cc494e9497f5.slice/crio-1d6071784185490dd2220067a9041d782f28dde5536c90d09e8b1f098fc188c0 WatchSource:0}: Error finding container 1d6071784185490dd2220067a9041d782f28dde5536c90d09e8b1f098fc188c0: Status 404 returned error can't find the container with id 1d6071784185490dd2220067a9041d782f28dde5536c90d09e8b1f098fc188c0
Jul 28 01:43:07.975605 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.975571    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt" event={"ID":"07ed1f71-d988-416a-9ee2-cc494e9497f5","Type":"ContainerStarted","Data":"1d6071784185490dd2220067a9041d782f28dde5536c90d09e8b1f098fc188c0"}
Jul 28 01:43:07.977372 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.977282    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52" event={"ID":"fde0d0e7-a727-44be-941c-8de65af76918","Type":"ContainerStarted","Data":"f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8"}
Jul 28 01:43:07.977372 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.977333    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52" event={"ID":"fde0d0e7-a727-44be-941c-8de65af76918","Type":"ContainerStarted","Data":"63dfdec809c228398455088f9edd87f17e85e6fae579a64f3396c17fd6e38b90"}
Jul 28 01:43:07.977508 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.977380    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:07.998809 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:07.998754    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52" podStartSLOduration=0.998737947 podStartE2EDuration="998.737947ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:07.996843462 +0000 UTC m=+2324.008082100" watchObservedRunningTime="2026-07-28 01:43:07.998737947 +0000 UTC m=+2324.009976555"
Jul 28 01:43:08.983356 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:08.983322    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt" event={"ID":"07ed1f71-d988-416a-9ee2-cc494e9497f5","Type":"ContainerStarted","Data":"7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6"}
Jul 28 01:43:08.983783 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:08.983414    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:09.000854 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:09.000794    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt" podStartSLOduration=2.000778788 podStartE2EDuration="2.000778788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:08.998782687 +0000 UTC m=+2325.010021315" watchObservedRunningTime="2026-07-28 01:43:09.000778788 +0000 UTC m=+2325.012017396"
Jul 28 01:43:10.037895 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:10.037856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:43:18.985598 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:18.985559    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:43:19.991125 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:19.991097    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:43:37.112754 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.112720    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"]
Jul 28 01:43:37.118736 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.116555    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.121117 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.121092    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-45a9e6\"/\"test-model-45a9e6-kserve-self-signed-certs\""
Jul 28 01:43:37.121289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.121090    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-45a9e6\"/\"kube-root-ca.crt\""
Jul 28 01:43:37.121289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.121147    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-45a9e6\"/\"default-dockercfg-xnp4q\""
Jul 28 01:43:37.121289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.121090    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-45a9e6\"/\"openshift-service-ca.crt\""
Jul 28 01:43:37.127999 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.127954    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"]
Jul 28 01:43:37.191064 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.191018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tls-certs\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.191268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.191080    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btqbp\" (UniqueName: \"kubernetes.io/projected/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-kube-api-access-btqbp\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.191268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.191106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-dshm\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.191268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.191125    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-model-cache\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.191268 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.191226    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tmp-dir\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.191420 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.191284    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-home\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.292319 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tls-certs\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.292545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btqbp\" (UniqueName: \"kubernetes.io/projected/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-kube-api-access-btqbp\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.292545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292366    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-dshm\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.292545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-model-cache\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.292545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292455    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tmp-dir\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.292545 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-home\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.293003 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292941    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-model-cache\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.293083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.293023    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tmp-dir\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.293083 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.292945    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-home\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.294886 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.294864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-dshm\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.295124 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.295108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tls-certs\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.301866 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.301844    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btqbp\" (UniqueName: \"kubernetes.io/projected/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-kube-api-access-btqbp\") pod \"test-model-45a9e6-kserve-67d55956df-bg8bz\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") " pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.429802 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.429705    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:37.565749 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:37.565720    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"]
Jul 28 01:43:37.567721 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:43:37.567696    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9419d31f_33a9_4e41_9e4c_7a59c0fb77fa.slice/crio-9ad7b28f3bb7fde7176b1287ae6c95b79983330c01fdb164d8ebf199cc298f69 WatchSource:0}: Error finding container 9ad7b28f3bb7fde7176b1287ae6c95b79983330c01fdb164d8ebf199cc298f69: Status 404 returned error can't find the container with id 9ad7b28f3bb7fde7176b1287ae6c95b79983330c01fdb164d8ebf199cc298f69
Jul 28 01:43:38.133259 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:38.133218    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz" event={"ID":"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa","Type":"ContainerStarted","Data":"1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55"}
Jul 28 01:43:38.133259 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:38.133267    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz" event={"ID":"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa","Type":"ContainerStarted","Data":"9ad7b28f3bb7fde7176b1287ae6c95b79983330c01fdb164d8ebf199cc298f69"}
Jul 28 01:43:38.150056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:38.150002    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz" podStartSLOduration=1.149954133 podStartE2EDuration="1.149954133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:43:38.149175869 +0000 UTC m=+2354.160414478" watchObservedRunningTime="2026-07-28 01:43:38.149954133 +0000 UTC m=+2354.161192743"
Jul 28 01:43:40.226550 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:40.226513    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:43:47.430647 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:47.430606    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:47.431183 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:47.430665    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:47.443179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:47.443152    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:48.199786 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:48.199750    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:43:54.335136 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:43:54.335097    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:44:04.628273 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:04.628231    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:44:16.540102 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:16.540061    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:44:40.802597 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:40.802562    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"]
Jul 28 01:44:41.872876 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:41.872841    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"]
Jul 28 01:44:41.876272 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:41.876255    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:41.878615 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:41.878592    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:44:41.878738 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:41.878619    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:44:41.883806 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:41.883779    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"]
Jul 28 01:44:41.971753 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:41.971708    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef40075a-a7a4-45c4-a4aa-99611ca4dacb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp\" (UID: \"ef40075a-a7a4-45c4-a4aa-99611ca4dacb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:42.073046 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:42.073001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef40075a-a7a4-45c4-a4aa-99611ca4dacb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp\" (UID: \"ef40075a-a7a4-45c4-a4aa-99611ca4dacb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:42.073638 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:42.073616    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef40075a-a7a4-45c4-a4aa-99611ca4dacb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp\" (UID: \"ef40075a-a7a4-45c4-a4aa-99611ca4dacb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:42.189027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:42.188903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:42.323085 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:42.323050    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"]
Jul 28 01:44:42.326142 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:44:42.326109    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef40075a_a7a4_45c4_a4aa_99611ca4dacb.slice/crio-d0dd45b08a3c11c7bbff7744fb2f5c49db8a40d74b0abbfc4ee220b12b3de4ef WatchSource:0}: Error finding container d0dd45b08a3c11c7bbff7744fb2f5c49db8a40d74b0abbfc4ee220b12b3de4ef: Status 404 returned error can't find the container with id d0dd45b08a3c11c7bbff7744fb2f5c49db8a40d74b0abbfc4ee220b12b3de4ef
Jul 28 01:44:42.461465 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:42.461370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp" event={"ID":"ef40075a-a7a4-45c4-a4aa-99611ca4dacb","Type":"ContainerStarted","Data":"d0dd45b08a3c11c7bbff7744fb2f5c49db8a40d74b0abbfc4ee220b12b3de4ef"}
Jul 28 01:44:43.467251 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:43.467212    2566 generic.go:356] "Generic (PLEG): container finished" podID="ef40075a-a7a4-45c4-a4aa-99611ca4dacb" containerID="4fd29be374f8007f491c5ac53efdafe640a537a46fa7284ddfb08ead5b28b111" exitCode=0
Jul 28 01:44:43.467692 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:43.467288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp" event={"ID":"ef40075a-a7a4-45c4-a4aa-99611ca4dacb","Type":"ContainerDied","Data":"4fd29be374f8007f491c5ac53efdafe640a537a46fa7284ddfb08ead5b28b111"}
Jul 28 01:44:44.604943 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:44.604913    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:44.699499 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:44.699462    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef40075a-a7a4-45c4-a4aa-99611ca4dacb-maas-api-service-ca\") pod \"ef40075a-a7a4-45c4-a4aa-99611ca4dacb\" (UID: \"ef40075a-a7a4-45c4-a4aa-99611ca4dacb\") "
Jul 28 01:44:44.699852 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:44.699827    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef40075a-a7a4-45c4-a4aa-99611ca4dacb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef40075a-a7a4-45c4-a4aa-99611ca4dacb" (UID: "ef40075a-a7a4-45c4-a4aa-99611ca4dacb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:44.800945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:44.800860    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef40075a-a7a4-45c4-a4aa-99611ca4dacb-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:45.478873 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:45.478831    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp" event={"ID":"ef40075a-a7a4-45c4-a4aa-99611ca4dacb","Type":"ContainerDied","Data":"d0dd45b08a3c11c7bbff7744fb2f5c49db8a40d74b0abbfc4ee220b12b3de4ef"}
Jul 28 01:44:45.478873 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:45.478879    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0dd45b08a3c11c7bbff7744fb2f5c49db8a40d74b0abbfc4ee220b12b3de4ef"
Jul 28 01:44:45.479118 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:45.478844    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"
Jul 28 01:44:46.031580 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:46.031549    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:44:52.523144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.523109    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"]
Jul 28 01:44:52.523570 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.523357    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9" podUID="dc53eda2-210c-4261-9c7b-b3a192597dc1" containerName="payload-processing" containerID="cri-o://920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f" gracePeriod=30
Jul 28 01:44:52.529828 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.529797    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"]
Jul 28 01:44:52.530078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.530052    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp" podUID="fef74a6a-182c-478e-a6ac-2e186c02227c" containerName="payload-pre-processing" containerID="cri-o://1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7" gracePeriod=30
Jul 28 01:44:52.706028 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.705990    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"]
Jul 28 01:44:52.708795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.708760    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-45a9e6-cvzzp"]
Jul 28 01:44:52.804811 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.804780    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:44:52.807550 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.807533    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:44:52.873009 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.872961    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8sm4\" (UniqueName: \"kubernetes.io/projected/dc53eda2-210c-4261-9c7b-b3a192597dc1-kube-api-access-x8sm4\") pod \"dc53eda2-210c-4261-9c7b-b3a192597dc1\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") "
Jul 28 01:44:52.873181 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.873047    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc53eda2-210c-4261-9c7b-b3a192597dc1-plugins-config-volume\") pod \"dc53eda2-210c-4261-9c7b-b3a192597dc1\" (UID: \"dc53eda2-210c-4261-9c7b-b3a192597dc1\") "
Jul 28 01:44:52.873181 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.873158    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fef74a6a-182c-478e-a6ac-2e186c02227c-plugins-config-volume\") pod \"fef74a6a-182c-478e-a6ac-2e186c02227c\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") "
Jul 28 01:44:52.873259 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.873195    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7q9l\" (UniqueName: \"kubernetes.io/projected/fef74a6a-182c-478e-a6ac-2e186c02227c-kube-api-access-s7q9l\") pod \"fef74a6a-182c-478e-a6ac-2e186c02227c\" (UID: \"fef74a6a-182c-478e-a6ac-2e186c02227c\") "
Jul 28 01:44:52.873499 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.873472    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc53eda2-210c-4261-9c7b-b3a192597dc1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dc53eda2-210c-4261-9c7b-b3a192597dc1" (UID: "dc53eda2-210c-4261-9c7b-b3a192597dc1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:52.873641 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.873516    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fef74a6a-182c-478e-a6ac-2e186c02227c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fef74a6a-182c-478e-a6ac-2e186c02227c" (UID: "fef74a6a-182c-478e-a6ac-2e186c02227c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:44:52.875312 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.875282    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc53eda2-210c-4261-9c7b-b3a192597dc1-kube-api-access-x8sm4" (OuterVolumeSpecName: "kube-api-access-x8sm4") pod "dc53eda2-210c-4261-9c7b-b3a192597dc1" (UID: "dc53eda2-210c-4261-9c7b-b3a192597dc1"). InnerVolumeSpecName "kube-api-access-x8sm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:52.875411 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.875331    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef74a6a-182c-478e-a6ac-2e186c02227c-kube-api-access-s7q9l" (OuterVolumeSpecName: "kube-api-access-s7q9l") pod "fef74a6a-182c-478e-a6ac-2e186c02227c" (UID: "fef74a6a-182c-478e-a6ac-2e186c02227c"). InnerVolumeSpecName "kube-api-access-s7q9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:52.974056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.974019    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7q9l\" (UniqueName: \"kubernetes.io/projected/fef74a6a-182c-478e-a6ac-2e186c02227c-kube-api-access-s7q9l\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:52.974056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.974050    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8sm4\" (UniqueName: \"kubernetes.io/projected/dc53eda2-210c-4261-9c7b-b3a192597dc1-kube-api-access-x8sm4\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:52.974056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.974060    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dc53eda2-210c-4261-9c7b-b3a192597dc1-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:52.974056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:52.974068    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fef74a6a-182c-478e-a6ac-2e186c02227c-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:53.519136 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.519093    2566 generic.go:356] "Generic (PLEG): container finished" podID="fef74a6a-182c-478e-a6ac-2e186c02227c" containerID="1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7" exitCode=0
Jul 28 01:44:53.519136 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.519139    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"
Jul 28 01:44:53.519405 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.519182    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp" event={"ID":"fef74a6a-182c-478e-a6ac-2e186c02227c","Type":"ContainerDied","Data":"1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7"}
Jul 28 01:44:53.519405 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.519239    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp" event={"ID":"fef74a6a-182c-478e-a6ac-2e186c02227c","Type":"ContainerDied","Data":"314864292da002a7fd9b49f0086cc2680ecbc9501bd66afc7d7c24990667940e"}
Jul 28 01:44:53.519405 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.519270    2566 scope.go:117] "RemoveContainer" containerID="1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7"
Jul 28 01:44:53.520637 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.520602    2566 generic.go:356] "Generic (PLEG): container finished" podID="dc53eda2-210c-4261-9c7b-b3a192597dc1" containerID="920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f" exitCode=0
Jul 28 01:44:53.520777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.520645    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"
Jul 28 01:44:53.520834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.520680    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9" event={"ID":"dc53eda2-210c-4261-9c7b-b3a192597dc1","Type":"ContainerDied","Data":"920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f"}
Jul 28 01:44:53.520879 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.520856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9" event={"ID":"dc53eda2-210c-4261-9c7b-b3a192597dc1","Type":"ContainerDied","Data":"1b5413229389f126e136bc6120b851c843a3f923aac28a3de86f0f43675dd901"}
Jul 28 01:44:53.529565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.529441    2566 scope.go:117] "RemoveContainer" containerID="1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7"
Jul 28 01:44:53.529800 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:44:53.529730    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7\": container with ID starting with 1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7 not found: ID does not exist" containerID="1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7"
Jul 28 01:44:53.529800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.529756    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7"} err="failed to get container status \"1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7\": rpc error: code = NotFound desc = could not find container \"1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7\": container with ID starting with 1c59009e3dc3b82fe56f32b507c4f7b3b4134a20d9737c40e628af809d54d7e7 not found: ID does not exist"
Jul 28 01:44:53.529800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.529772    2566 scope.go:117] "RemoveContainer" containerID="920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f"
Jul 28 01:44:53.538125 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.538108    2566 scope.go:117] "RemoveContainer" containerID="920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f"
Jul 28 01:44:53.538378 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:44:53.538358    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f\": container with ID starting with 920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f not found: ID does not exist" containerID="920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f"
Jul 28 01:44:53.538445 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.538387    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f"} err="failed to get container status \"920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f\": rpc error: code = NotFound desc = could not find container \"920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f\": container with ID starting with 920ad3442f89cd41066a879b30bea7701efc1cce89cbe605342a0351258b8b4f not found: ID does not exist"
Jul 28 01:44:53.543951 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.543925    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"]
Jul 28 01:44:53.547788 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.547769    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-45a9e6-666575b7f8-wblk9"]
Jul 28 01:44:53.561080 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.561051    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"]
Jul 28 01:44:53.563771 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:53.563749    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-45a9e6-5496f89844-248mp"]
Jul 28 01:44:54.559880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:54.559843    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc53eda2-210c-4261-9c7b-b3a192597dc1" path="/var/lib/kubelet/pods/dc53eda2-210c-4261-9c7b-b3a192597dc1/volumes"
Jul 28 01:44:54.560280 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:54.560213    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef40075a-a7a4-45c4-a4aa-99611ca4dacb" path="/var/lib/kubelet/pods/ef40075a-a7a4-45c4-a4aa-99611ca4dacb/volumes"
Jul 28 01:44:54.560529 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:54.560516    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fef74a6a-182c-478e-a6ac-2e186c02227c" path="/var/lib/kubelet/pods/fef74a6a-182c-478e-a6ac-2e186c02227c/volumes"
Jul 28 01:44:55.822641 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:55.822600    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz" podUID="9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" containerName="main" containerID="cri-o://1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55" gracePeriod=45
Jul 28 01:44:56.081992 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.081903    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:44:56.100551 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.100519    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-model-cache\") pod \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") "
Jul 28 01:44:56.100745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.100570    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btqbp\" (UniqueName: \"kubernetes.io/projected/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-kube-api-access-btqbp\") pod \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") "
Jul 28 01:44:56.100745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.100596    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-dshm\") pod \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") "
Jul 28 01:44:56.100745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.100712    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tls-certs\") pod \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") "
Jul 28 01:44:56.100745 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.100742    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tmp-dir\") pod \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") "
Jul 28 01:44:56.100953 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.100776    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-home\") pod \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\" (UID: \"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa\") "
Jul 28 01:44:56.101276 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.101157    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" (UID: "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.101276 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.101225    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-home" (OuterVolumeSpecName: "home") pod "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" (UID: "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.101276 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.101239    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" (UID: "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.103264 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.103234    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-dshm" (OuterVolumeSpecName: "dshm") pod "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" (UID: "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:44:56.103572 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.103544    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-kube-api-access-btqbp" (OuterVolumeSpecName: "kube-api-access-btqbp") pod "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" (UID: "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa"). InnerVolumeSpecName "kube-api-access-btqbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:44:56.103678 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.103591    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" (UID: "9419d31f-33a9-4e41-9e4c-7a59c0fb77fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:44:56.201408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.201364    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.201408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.201404    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.201408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.201413    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.201408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.201421    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btqbp\" (UniqueName: \"kubernetes.io/projected/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-kube-api-access-btqbp\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.201746 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.201430    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.201746 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.201438    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:44:56.544573 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.544537    2566 generic.go:356] "Generic (PLEG): container finished" podID="9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" containerID="1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55" exitCode=0
Jul 28 01:44:56.544780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.544585    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"
Jul 28 01:44:56.544780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.544595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz" event={"ID":"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa","Type":"ContainerDied","Data":"1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55"}
Jul 28 01:44:56.544780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.544650    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz" event={"ID":"9419d31f-33a9-4e41-9e4c-7a59c0fb77fa","Type":"ContainerDied","Data":"9ad7b28f3bb7fde7176b1287ae6c95b79983330c01fdb164d8ebf199cc298f69"}
Jul 28 01:44:56.544780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.544674    2566 scope.go:117] "RemoveContainer" containerID="1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55"
Jul 28 01:44:56.554040 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.554023    2566 scope.go:117] "RemoveContainer" containerID="1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55"
Jul 28 01:44:56.554282 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:44:56.554265    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55\": container with ID starting with 1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55 not found: ID does not exist" containerID="1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55"
Jul 28 01:44:56.554359 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.554288    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55"} err="failed to get container status \"1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55\": rpc error: code = NotFound desc = could not find container \"1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55\": container with ID starting with 1ac966ffd983b76514ffacad8c817fb2156d8d9cd14bee5413d0c3e422de9e55 not found: ID does not exist"
Jul 28 01:44:56.565549 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.565528    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"]
Jul 28 01:44:56.568805 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:56.568785    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-45a9e6/test-model-45a9e6-kserve-67d55956df-bg8bz"]
Jul 28 01:44:58.134624 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:58.134588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:44:58.560263 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:58.560186    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" path="/var/lib/kubelet/pods/9419d31f-33a9-4e41-9e4c-7a59c0fb77fa/volumes"
Jul 28 01:44:59.569482 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569447    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"]
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569767    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc53eda2-210c-4261-9c7b-b3a192597dc1" containerName="payload-processing"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569779    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc53eda2-210c-4261-9c7b-b3a192597dc1" containerName="payload-processing"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569788    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef74a6a-182c-478e-a6ac-2e186c02227c" containerName="payload-pre-processing"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569794    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef74a6a-182c-478e-a6ac-2e186c02227c" containerName="payload-pre-processing"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569801    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef40075a-a7a4-45c4-a4aa-99611ca4dacb" containerName="revoke-keys"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569807    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef40075a-a7a4-45c4-a4aa-99611ca4dacb" containerName="revoke-keys"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569824    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" containerName="main"
Jul 28 01:44:59.569864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569829    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" containerName="main"
Jul 28 01:44:59.570159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569884    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef40075a-a7a4-45c4-a4aa-99611ca4dacb" containerName="revoke-keys"
Jul 28 01:44:59.570159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569893    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc53eda2-210c-4261-9c7b-b3a192597dc1" containerName="payload-processing"
Jul 28 01:44:59.570159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569899    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9419d31f-33a9-4e41-9e4c-7a59c0fb77fa" containerName="main"
Jul 28 01:44:59.570159 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.569906    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef74a6a-182c-478e-a6ac-2e186c02227c" containerName="payload-pre-processing"
Jul 28 01:44:59.572923 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.572905    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:44:59.575323 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.575296    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:44:59.575472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.575333    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:44:59.580708 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.580106    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"]
Jul 28 01:44:59.633960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.633921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98931dad-1b6b-4538-a814-30a14a68d308-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2\" (UID: \"98931dad-1b6b-4538-a814-30a14a68d308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:44:59.735229 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.735195    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98931dad-1b6b-4538-a814-30a14a68d308-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2\" (UID: \"98931dad-1b6b-4538-a814-30a14a68d308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:44:59.735869 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.735850    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98931dad-1b6b-4538-a814-30a14a68d308-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2\" (UID: \"98931dad-1b6b-4538-a814-30a14a68d308\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:44:59.884816 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:44:59.884778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:45:00.030948 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.030914    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"]
Jul 28 01:45:00.034515 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:00.034490    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98931dad_1b6b_4538_a814_30a14a68d308.slice/crio-b47d6535d0bd41733c86be0db301b83f7b1f0b72d899e2ba7e31830df8571303 WatchSource:0}: Error finding container b47d6535d0bd41733c86be0db301b83f7b1f0b72d899e2ba7e31830df8571303: Status 404 returned error can't find the container with id b47d6535d0bd41733c86be0db301b83f7b1f0b72d899e2ba7e31830df8571303
Jul 28 01:45:00.170827 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.170741    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"]
Jul 28 01:45:00.174880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.174862    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"
Jul 28 01:45:00.177659 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.177634    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"]
Jul 28 01:45:00.180650 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.180632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"
Jul 28 01:45:00.184752 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.184731    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"
Jul 28 01:45:00.190266 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.190240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"]
Jul 28 01:45:00.194135 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.194104    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"]
Jul 28 01:45:00.195611 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.195509    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"
Jul 28 01:45:00.378298 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.378267    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"]
Jul 28 01:45:00.381653 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:00.381626    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f8fd4e_9383_401b_938b_fc42839573aa.slice/crio-d40bfeee908dcba4abb17aa1e41d0f84a889881358bbe6b405de1843108c4c62 WatchSource:0}: Error finding container d40bfeee908dcba4abb17aa1e41d0f84a889881358bbe6b405de1843108c4c62: Status 404 returned error can't find the container with id d40bfeee908dcba4abb17aa1e41d0f84a889881358bbe6b405de1843108c4c62
Jul 28 01:45:00.546168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.546120    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"]
Jul 28 01:45:00.560076 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:00.560050    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c69332b_61f4_4438_8b78_ffc38adab825.slice/crio-791b795e84063c67e059bf3162ac338d0fa94a9b9811e05caa8cd93fb383060d WatchSource:0}: Error finding container 791b795e84063c67e059bf3162ac338d0fa94a9b9811e05caa8cd93fb383060d: Status 404 returned error can't find the container with id 791b795e84063c67e059bf3162ac338d0fa94a9b9811e05caa8cd93fb383060d
Jul 28 01:45:00.565486 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.565455    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2" event={"ID":"98931dad-1b6b-4538-a814-30a14a68d308","Type":"ContainerStarted","Data":"5295272efabb2b24eb46c13ec0ca17460bb4eeaba65b6c8cf68f9b3d90f3dd1b"}
Jul 28 01:45:00.565581 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.565500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2" event={"ID":"98931dad-1b6b-4538-a814-30a14a68d308","Type":"ContainerStarted","Data":"b47d6535d0bd41733c86be0db301b83f7b1f0b72d899e2ba7e31830df8571303"}
Jul 28 01:45:00.566689 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.566671    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht" event={"ID":"a5f8fd4e-9383-401b-938b-fc42839573aa","Type":"ContainerStarted","Data":"d40bfeee908dcba4abb17aa1e41d0f84a889881358bbe6b405de1843108c4c62"}
Jul 28 01:45:00.587498 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:00.587439    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2" podStartSLOduration=1.587409832 podStartE2EDuration="1.587409832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:44:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:00.587191027 +0000 UTC m=+2436.598429639" watchObservedRunningTime="2026-07-28 01:45:00.587409832 +0000 UTC m=+2436.598648443"
Jul 28 01:45:01.573200 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.573159    2566 generic.go:356] "Generic (PLEG): container finished" podID="6c69332b-61f4-4438-8b78-ffc38adab825" containerID="b931199fdb3d139f34cd3c780de3621537f5a7f9a932eed79151491af4729237" exitCode=0
Jul 28 01:45:01.573400 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.573265    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2" event={"ID":"6c69332b-61f4-4438-8b78-ffc38adab825","Type":"ContainerDied","Data":"b931199fdb3d139f34cd3c780de3621537f5a7f9a932eed79151491af4729237"}
Jul 28 01:45:01.573400 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.573309    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2" event={"ID":"6c69332b-61f4-4438-8b78-ffc38adab825","Type":"ContainerStarted","Data":"791b795e84063c67e059bf3162ac338d0fa94a9b9811e05caa8cd93fb383060d"}
Jul 28 01:45:01.574689 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.574665    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5f8fd4e-9383-401b-938b-fc42839573aa" containerID="6afd0e7d4316a79089c2eb38df3fc45767dc14cfdf6802a5c8dcdb0cf39673f0" exitCode=0
Jul 28 01:45:01.574763 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.574745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht" event={"ID":"a5f8fd4e-9383-401b-938b-fc42839573aa","Type":"ContainerDied","Data":"6afd0e7d4316a79089c2eb38df3fc45767dc14cfdf6802a5c8dcdb0cf39673f0"}
Jul 28 01:45:01.576104 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.576082    2566 generic.go:356] "Generic (PLEG): container finished" podID="98931dad-1b6b-4538-a814-30a14a68d308" containerID="5295272efabb2b24eb46c13ec0ca17460bb4eeaba65b6c8cf68f9b3d90f3dd1b" exitCode=0
Jul 28 01:45:01.576182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:01.576162    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2" event={"ID":"98931dad-1b6b-4538-a814-30a14a68d308","Type":"ContainerDied","Data":"5295272efabb2b24eb46c13ec0ca17460bb4eeaba65b6c8cf68f9b3d90f3dd1b"}
Jul 28 01:45:02.736718 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:02.736688    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:45:02.763649 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:02.763613    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98931dad-1b6b-4538-a814-30a14a68d308-maas-api-service-ca\") pod \"98931dad-1b6b-4538-a814-30a14a68d308\" (UID: \"98931dad-1b6b-4538-a814-30a14a68d308\") "
Jul 28 01:45:02.764049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:02.764023    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98931dad-1b6b-4538-a814-30a14a68d308-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "98931dad-1b6b-4538-a814-30a14a68d308" (UID: "98931dad-1b6b-4538-a814-30a14a68d308"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:45:02.795501 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:02.795475    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"
Jul 28 01:45:02.798459 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:02.798437    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"
Jul 28 01:45:02.864729 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:02.864699    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/98931dad-1b6b-4538-a814-30a14a68d308-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:45:03.586054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.586021    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"
Jul 28 01:45:03.586054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.586034    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2" event={"ID":"98931dad-1b6b-4538-a814-30a14a68d308","Type":"ContainerDied","Data":"b47d6535d0bd41733c86be0db301b83f7b1f0b72d899e2ba7e31830df8571303"}
Jul 28 01:45:03.586447 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.586080    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b47d6535d0bd41733c86be0db301b83f7b1f0b72d899e2ba7e31830df8571303"
Jul 28 01:45:03.587535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.587512    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"
Jul 28 01:45:03.587643 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.587515    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2" event={"ID":"6c69332b-61f4-4438-8b78-ffc38adab825","Type":"ContainerDied","Data":"791b795e84063c67e059bf3162ac338d0fa94a9b9811e05caa8cd93fb383060d"}
Jul 28 01:45:03.587712 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.587645    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="791b795e84063c67e059bf3162ac338d0fa94a9b9811e05caa8cd93fb383060d"
Jul 28 01:45:03.588998 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.588949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht" event={"ID":"a5f8fd4e-9383-401b-938b-fc42839573aa","Type":"ContainerDied","Data":"d40bfeee908dcba4abb17aa1e41d0f84a889881358bbe6b405de1843108c4c62"}
Jul 28 01:45:03.589095 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.589008    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d40bfeee908dcba4abb17aa1e41d0f84a889881358bbe6b405de1843108c4c62"
Jul 28 01:45:03.589095 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:03.589034    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753385-k4jht"
Jul 28 01:45:10.181374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.181332    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"]
Jul 28 01:45:10.182924 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.182900    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-a5bff2-29753385-j2vt2"]
Jul 28 01:45:10.213927 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.213886    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"]
Jul 28 01:45:10.214192 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.214163    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52" podUID="fde0d0e7-a727-44be-941c-8de65af76918" containerName="payload-processing" containerID="cri-o://f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8" gracePeriod=30
Jul 28 01:45:10.218939 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.218886    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"]
Jul 28 01:45:10.219374 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.219343    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt" podUID="07ed1f71-d988-416a-9ee2-cc494e9497f5" containerName="payload-pre-processing" containerID="cri-o://7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6" gracePeriod=30
Jul 28 01:45:10.363154 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.363114    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"]
Jul 28 01:45:10.366576 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.366547    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a5bff2-w5hg2"]
Jul 28 01:45:10.497175 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.497147    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:45:10.500119 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.500097    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:45:10.530906 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.530816    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkn6q\" (UniqueName: \"kubernetes.io/projected/fde0d0e7-a727-44be-941c-8de65af76918-kube-api-access-bkn6q\") pod \"fde0d0e7-a727-44be-941c-8de65af76918\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") "
Jul 28 01:45:10.530906 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.530873    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fde0d0e7-a727-44be-941c-8de65af76918-plugins-config-volume\") pod \"fde0d0e7-a727-44be-941c-8de65af76918\" (UID: \"fde0d0e7-a727-44be-941c-8de65af76918\") "
Jul 28 01:45:10.531585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.531555    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fde0d0e7-a727-44be-941c-8de65af76918-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fde0d0e7-a727-44be-941c-8de65af76918" (UID: "fde0d0e7-a727-44be-941c-8de65af76918"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:45:10.534051 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.534019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde0d0e7-a727-44be-941c-8de65af76918-kube-api-access-bkn6q" (OuterVolumeSpecName: "kube-api-access-bkn6q") pod "fde0d0e7-a727-44be-941c-8de65af76918" (UID: "fde0d0e7-a727-44be-941c-8de65af76918"). InnerVolumeSpecName "kube-api-access-bkn6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:45:10.561611 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.561582    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c69332b-61f4-4438-8b78-ffc38adab825" path="/var/lib/kubelet/pods/6c69332b-61f4-4438-8b78-ffc38adab825/volumes"
Jul 28 01:45:10.561938 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.561921    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98931dad-1b6b-4538-a814-30a14a68d308" path="/var/lib/kubelet/pods/98931dad-1b6b-4538-a814-30a14a68d308/volumes"
Jul 28 01:45:10.624593 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.624552    2566 generic.go:356] "Generic (PLEG): container finished" podID="07ed1f71-d988-416a-9ee2-cc494e9497f5" containerID="7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6" exitCode=0
Jul 28 01:45:10.624593 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.624599    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"
Jul 28 01:45:10.624826 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.624638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt" event={"ID":"07ed1f71-d988-416a-9ee2-cc494e9497f5","Type":"ContainerDied","Data":"7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6"}
Jul 28 01:45:10.624826 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.624702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt" event={"ID":"07ed1f71-d988-416a-9ee2-cc494e9497f5","Type":"ContainerDied","Data":"1d6071784185490dd2220067a9041d782f28dde5536c90d09e8b1f098fc188c0"}
Jul 28 01:45:10.624826 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.624736    2566 scope.go:117] "RemoveContainer" containerID="7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6"
Jul 28 01:45:10.626007 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.625958    2566 generic.go:356] "Generic (PLEG): container finished" podID="fde0d0e7-a727-44be-941c-8de65af76918" containerID="f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8" exitCode=0
Jul 28 01:45:10.626007 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.625996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52" event={"ID":"fde0d0e7-a727-44be-941c-8de65af76918","Type":"ContainerDied","Data":"f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8"}
Jul 28 01:45:10.626196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.626012    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"
Jul 28 01:45:10.626196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.626037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52" event={"ID":"fde0d0e7-a727-44be-941c-8de65af76918","Type":"ContainerDied","Data":"63dfdec809c228398455088f9edd87f17e85e6fae579a64f3396c17fd6e38b90"}
Jul 28 01:45:10.632187 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.632162    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7xzr\" (UniqueName: \"kubernetes.io/projected/07ed1f71-d988-416a-9ee2-cc494e9497f5-kube-api-access-k7xzr\") pod \"07ed1f71-d988-416a-9ee2-cc494e9497f5\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") "
Jul 28 01:45:10.632343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.632239    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ed1f71-d988-416a-9ee2-cc494e9497f5-plugins-config-volume\") pod \"07ed1f71-d988-416a-9ee2-cc494e9497f5\" (UID: \"07ed1f71-d988-416a-9ee2-cc494e9497f5\") "
Jul 28 01:45:10.632536 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.632519    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkn6q\" (UniqueName: \"kubernetes.io/projected/fde0d0e7-a727-44be-941c-8de65af76918-kube-api-access-bkn6q\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.632609 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.632541    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fde0d0e7-a727-44be-941c-8de65af76918-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.632647 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.632603    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07ed1f71-d988-416a-9ee2-cc494e9497f5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "07ed1f71-d988-416a-9ee2-cc494e9497f5" (UID: "07ed1f71-d988-416a-9ee2-cc494e9497f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:45:10.633834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.633819    2566 scope.go:117] "RemoveContainer" containerID="7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6"
Jul 28 01:45:10.634138 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:45:10.634117    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6\": container with ID starting with 7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6 not found: ID does not exist" containerID="7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6"
Jul 28 01:45:10.634208 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.634145    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6"} err="failed to get container status \"7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6\": rpc error: code = NotFound desc = could not find container \"7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6\": container with ID starting with 7bcaf6948e27922452ec90d039ad035a955a926ac3295df5610036dc8e3e45d6 not found: ID does not exist"
Jul 28 01:45:10.634208 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.634162    2566 scope.go:117] "RemoveContainer" containerID="f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8"
Jul 28 01:45:10.634381 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.634358    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ed1f71-d988-416a-9ee2-cc494e9497f5-kube-api-access-k7xzr" (OuterVolumeSpecName: "kube-api-access-k7xzr") pod "07ed1f71-d988-416a-9ee2-cc494e9497f5" (UID: "07ed1f71-d988-416a-9ee2-cc494e9497f5"). InnerVolumeSpecName "kube-api-access-k7xzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:45:10.642608 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.642588    2566 scope.go:117] "RemoveContainer" containerID="f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8"
Jul 28 01:45:10.642898 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:45:10.642874    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8\": container with ID starting with f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8 not found: ID does not exist" containerID="f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8"
Jul 28 01:45:10.642946 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.642910    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8"} err="failed to get container status \"f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8\": rpc error: code = NotFound desc = could not find container \"f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8\": container with ID starting with f1db643ff536cce8dc268153f17f28e05361bc6efe1942d2779a1a3b08b75ec8 not found: ID does not exist"
Jul 28 01:45:10.643027 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.643014    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"]
Jul 28 01:45:10.646694 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.646666    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a5bff2-5c4585d896-vbc52"]
Jul 28 01:45:10.733764 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.733674    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7xzr\" (UniqueName: \"kubernetes.io/projected/07ed1f71-d988-416a-9ee2-cc494e9497f5-kube-api-access-k7xzr\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.733764 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.733703    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/07ed1f71-d988-416a-9ee2-cc494e9497f5-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:45:10.946065 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.946034    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"]
Jul 28 01:45:10.951372 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:10.951341    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a5bff2-64d9c84888-bpvqt"]
Jul 28 01:45:12.560297 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:12.560263    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07ed1f71-d988-416a-9ee2-cc494e9497f5" path="/var/lib/kubelet/pods/07ed1f71-d988-416a-9ee2-cc494e9497f5/volumes"
Jul 28 01:45:12.560802 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:12.560765    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fde0d0e7-a727-44be-941c-8de65af76918" path="/var/lib/kubelet/pods/fde0d0e7-a727-44be-941c-8de65af76918/volumes"
Jul 28 01:45:13.237614 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:13.237574    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:45:20.932779 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.932744    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"]
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933069    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c69332b-61f4-4438-8b78-ffc38adab825" containerName="cleanup"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933080    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c69332b-61f4-4438-8b78-ffc38adab825" containerName="cleanup"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933092    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fde0d0e7-a727-44be-941c-8de65af76918" containerName="payload-processing"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933099    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fde0d0e7-a727-44be-941c-8de65af76918" containerName="payload-processing"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933105    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ed1f71-d988-416a-9ee2-cc494e9497f5" containerName="payload-pre-processing"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933111    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ed1f71-d988-416a-9ee2-cc494e9497f5" containerName="payload-pre-processing"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933122    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98931dad-1b6b-4538-a814-30a14a68d308" containerName="revoke-keys"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933127    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="98931dad-1b6b-4538-a814-30a14a68d308" containerName="revoke-keys"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933133    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5f8fd4e-9383-401b-938b-fc42839573aa" containerName="cleanup"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933139    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5f8fd4e-9383-401b-938b-fc42839573aa" containerName="cleanup"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933184    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c69332b-61f4-4438-8b78-ffc38adab825" containerName="cleanup"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933190    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="98931dad-1b6b-4538-a814-30a14a68d308" containerName="revoke-keys"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933198    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fde0d0e7-a727-44be-941c-8de65af76918" containerName="payload-processing"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933205    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5f8fd4e-9383-401b-938b-fc42839573aa" containerName="cleanup"
Jul 28 01:45:20.933212 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.933212    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ed1f71-d988-416a-9ee2-cc494e9497f5" containerName="payload-pre-processing"
Jul 28 01:45:20.937847 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.937826    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:20.940182 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.940158    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ea62b7\""
Jul 28 01:45:20.940293 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.940195    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ea62b7-dockercfg-92lg9\""
Jul 28 01:45:20.946585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.946561    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"]
Jul 28 01:45:20.965244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.965212    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"]
Jul 28 01:45:20.968557 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.968540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:20.982925 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:20.982898    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"]
Jul 28 01:45:21.023611 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.023569    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc49fa2-269e-4346-a3cd-0f7da71d6443-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.023795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.023622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5p6w\" (UniqueName: \"kubernetes.io/projected/bbc49fa2-269e-4346-a3cd-0f7da71d6443-kube-api-access-l5p6w\") pod \"payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.124725 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.124683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc49fa2-269e-4346-a3cd-0f7da71d6443-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.124941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.124740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5p6w\" (UniqueName: \"kubernetes.io/projected/bbc49fa2-269e-4346-a3cd-0f7da71d6443-kube-api-access-l5p6w\") pod \"payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.124941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.124788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.124941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.124819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbhs6\" (UniqueName: \"kubernetes.io/projected/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-kube-api-access-rbhs6\") pod \"payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.125441 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.125422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc49fa2-269e-4346-a3cd-0f7da71d6443-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.134611 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.134581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5p6w\" (UniqueName: \"kubernetes.io/projected/bbc49fa2-269e-4346-a3cd-0f7da71d6443-kube-api-access-l5p6w\") pod \"payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.225610 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.225517    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.225610 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.225570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbhs6\" (UniqueName: \"kubernetes.io/projected/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-kube-api-access-rbhs6\") pod \"payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.226250 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.226231    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.236050 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.236021    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbhs6\" (UniqueName: \"kubernetes.io/projected/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-kube-api-access-rbhs6\") pod \"payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") " pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.248880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.248860    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.278184 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.278148    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.401301 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.401235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"]
Jul 28 01:45:21.404925 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:21.404898    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc49fa2_269e_4346_a3cd_0f7da71d6443.slice/crio-5dda159f286a39a72b6afe7aa32424b82be9b71b075d3ab0c4d9e532a87b3090 WatchSource:0}: Error finding container 5dda159f286a39a72b6afe7aa32424b82be9b71b075d3ab0c4d9e532a87b3090: Status 404 returned error can't find the container with id 5dda159f286a39a72b6afe7aa32424b82be9b71b075d3ab0c4d9e532a87b3090
Jul 28 01:45:21.434407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.434381    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"]
Jul 28 01:45:21.438193 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:21.438164    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a4da9c_4df1_45b5_9d77_538754ad9e7a.slice/crio-4bf9ac80468045cf5fe90b0e3199cfa8da07774297fd19f1558a0b20cf57da8c WatchSource:0}: Error finding container 4bf9ac80468045cf5fe90b0e3199cfa8da07774297fd19f1558a0b20cf57da8c: Status 404 returned error can't find the container with id 4bf9ac80468045cf5fe90b0e3199cfa8da07774297fd19f1558a0b20cf57da8c
Jul 28 01:45:21.682209 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.682176    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl" event={"ID":"bbc49fa2-269e-4346-a3cd-0f7da71d6443","Type":"ContainerStarted","Data":"e34c55744ca2c3c34bfc5488518f1fc7f14ccf492ce87f42e05da2bc738e6090"}
Jul 28 01:45:21.682429 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.682234    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl" event={"ID":"bbc49fa2-269e-4346-a3cd-0f7da71d6443","Type":"ContainerStarted","Data":"5dda159f286a39a72b6afe7aa32424b82be9b71b075d3ab0c4d9e532a87b3090"}
Jul 28 01:45:21.682429 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.682359    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:21.683756 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.683728    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8" event={"ID":"b2a4da9c-4df1-45b5-9d77-538754ad9e7a","Type":"ContainerStarted","Data":"897216f45929beacdb5e5b32a39a3e56f64fbc3ab93fd392acbbb5d32c8b759f"}
Jul 28 01:45:21.683885 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.683778    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8" event={"ID":"b2a4da9c-4df1-45b5-9d77-538754ad9e7a","Type":"ContainerStarted","Data":"4bf9ac80468045cf5fe90b0e3199cfa8da07774297fd19f1558a0b20cf57da8c"}
Jul 28 01:45:21.683930 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.683882    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:21.698600 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.698540    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl" podStartSLOduration=1.698522233 podStartE2EDuration="1.698522233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:21.697938156 +0000 UTC m=+2457.709176766" watchObservedRunningTime="2026-07-28 01:45:21.698522233 +0000 UTC m=+2457.709760844"
Jul 28 01:45:21.717598 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:21.716922    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8" podStartSLOduration=1.7169081240000001 podStartE2EDuration="1.716908124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:21.71656143 +0000 UTC m=+2457.727800040" watchObservedRunningTime="2026-07-28 01:45:21.716908124 +0000 UTC m=+2457.728146732"
Jul 28 01:45:24.758585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:24.758555    2566 scope.go:117] "RemoveContainer" containerID="8491adb227ac20828800e4aacbb0d923378b06d9093c471102dc28a39bb17215"
Jul 28 01:45:25.242406 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:25.242369    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:45:32.691633 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:32.691602    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:45:32.692168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:32.691671    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:45:50.261863 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.261826    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"]
Jul 28 01:45:50.265583 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.265565    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.268226 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.268203    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-5e7618\""
Jul 28 01:45:50.268341 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.268222    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-5e7618-dockercfg-8jbzs\""
Jul 28 01:45:50.274309 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.274285    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"]
Jul 28 01:45:50.324865 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.324824    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"]
Jul 28 01:45:50.328196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.328176    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.338925 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.338897    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"]
Jul 28 01:45:50.383361 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.383324    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-plugins-config-volume\") pod \"payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") " pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.383558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.383376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fmkm\" (UniqueName: \"kubernetes.io/projected/957056c4-613e-4a20-ace4-4fb0e3e50566-kube-api-access-7fmkm\") pod \"payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.383558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.383477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957056c4-613e-4a20-ace4-4fb0e3e50566-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.383558 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.383532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzsgk\" (UniqueName: \"kubernetes.io/projected/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-kube-api-access-jzsgk\") pod \"payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") " pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.484209 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.484174    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-plugins-config-volume\") pod \"payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") " pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.484396 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.484227    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fmkm\" (UniqueName: \"kubernetes.io/projected/957056c4-613e-4a20-ace4-4fb0e3e50566-kube-api-access-7fmkm\") pod \"payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.484396 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.484279    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957056c4-613e-4a20-ace4-4fb0e3e50566-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.484527 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.484436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzsgk\" (UniqueName: \"kubernetes.io/projected/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-kube-api-access-jzsgk\") pod \"payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") " pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.485056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.485009    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-plugins-config-volume\") pod \"payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") " pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.485195 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.485138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957056c4-613e-4a20-ace4-4fb0e3e50566-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.492803 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.492780    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzsgk\" (UniqueName: \"kubernetes.io/projected/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-kube-api-access-jzsgk\") pod \"payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") " pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.492951 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.492888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fmkm\" (UniqueName: \"kubernetes.io/projected/957056c4-613e-4a20-ace4-4fb0e3e50566-kube-api-access-7fmkm\") pod \"payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.576864 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.576769    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.638756 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.638723    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:50.727546 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.727455    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"]
Jul 28 01:45:50.731636 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:50.731593    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd6f766_b8d0_4f1b_a52b_95f8eee6deff.slice/crio-4c5fe9ed70cb63dfd81654e54d0db45f6c499c0a301f8d12b84d8371a5a568e8 WatchSource:0}: Error finding container 4c5fe9ed70cb63dfd81654e54d0db45f6c499c0a301f8d12b84d8371a5a568e8: Status 404 returned error can't find the container with id 4c5fe9ed70cb63dfd81654e54d0db45f6c499c0a301f8d12b84d8371a5a568e8
Jul 28 01:45:50.793408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.793385    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"]
Jul 28 01:45:50.795641 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:45:50.795617    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957056c4_613e_4a20_ace4_4fb0e3e50566.slice/crio-271a6ce988d160f7ed24fecda3031d1ab5346ee857e3a1b48d938f3dfbd7556c WatchSource:0}: Error finding container 271a6ce988d160f7ed24fecda3031d1ab5346ee857e3a1b48d938f3dfbd7556c: Status 404 returned error can't find the container with id 271a6ce988d160f7ed24fecda3031d1ab5346ee857e3a1b48d938f3dfbd7556c
Jul 28 01:45:50.825890 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.825859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg" event={"ID":"957056c4-613e-4a20-ace4-4fb0e3e50566","Type":"ContainerStarted","Data":"271a6ce988d160f7ed24fecda3031d1ab5346ee857e3a1b48d938f3dfbd7556c"}
Jul 28 01:45:50.827395 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.827326    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb" event={"ID":"acd6f766-b8d0-4f1b-a52b-95f8eee6deff","Type":"ContainerStarted","Data":"17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585"}
Jul 28 01:45:50.827395 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.827377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb" event={"ID":"acd6f766-b8d0-4f1b-a52b-95f8eee6deff","Type":"ContainerStarted","Data":"4c5fe9ed70cb63dfd81654e54d0db45f6c499c0a301f8d12b84d8371a5a568e8"}
Jul 28 01:45:50.827556 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.827408    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:45:50.843263 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:50.843200    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb" podStartSLOduration=0.843180225 podStartE2EDuration="843.180225ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:50.842706666 +0000 UTC m=+2486.853945276" watchObservedRunningTime="2026-07-28 01:45:50.843180225 +0000 UTC m=+2486.854418847"
Jul 28 01:45:51.833721 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:51.833684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg" event={"ID":"957056c4-613e-4a20-ace4-4fb0e3e50566","Type":"ContainerStarted","Data":"d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e"}
Jul 28 01:45:51.834171 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:51.833813    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:45:51.851406 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:51.851350    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg" podStartSLOduration=1.8513337920000001 podStartE2EDuration="1.851333792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:45:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:45:51.848536537 +0000 UTC m=+2487.859775156" watchObservedRunningTime="2026-07-28 01:45:51.851333792 +0000 UTC m=+2487.862572400"
Jul 28 01:45:56.528534 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:45:56.528493    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:46:01.835806 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:01.835772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:46:02.841292 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:02.841260    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:46:17.353230 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.353190    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"]
Jul 28 01:46:17.356810 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.356789    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:17.359407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.359384    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:46:17.359407 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.359395    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:46:17.369550 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.369524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"]
Jul 28 01:46:17.533245 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.533210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ea62b7-779bn\" (UID: \"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:17.634011 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.633898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ea62b7-779bn\" (UID: \"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:17.634532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.634513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-ea62b7-779bn\" (UID: \"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:17.667110 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.667078    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:17.807678 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.807649    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"]
Jul 28 01:46:17.809911 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:46:17.809880    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61b3d53_f3aa_44e8_b2d9_021e5e325ffd.slice/crio-267f786d025938790a510bfe4d9272cf3150356a687a866756b536fe8d7cd6f4 WatchSource:0}: Error finding container 267f786d025938790a510bfe4d9272cf3150356a687a866756b536fe8d7cd6f4: Status 404 returned error can't find the container with id 267f786d025938790a510bfe4d9272cf3150356a687a866756b536fe8d7cd6f4
Jul 28 01:46:17.962372 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:17.962279    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn" event={"ID":"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd","Type":"ContainerStarted","Data":"267f786d025938790a510bfe4d9272cf3150356a687a866756b536fe8d7cd6f4"}
Jul 28 01:46:18.968285 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:18.968253    2566 generic.go:356] "Generic (PLEG): container finished" podID="c61b3d53-f3aa-44e8-b2d9-021e5e325ffd" containerID="a5faaf0ee703ef619fb48c6e054f3ef246a916d79906adef09a1ad9254d7abc5" exitCode=0
Jul 28 01:46:18.968688 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:18.968333    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn" event={"ID":"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd","Type":"ContainerDied","Data":"a5faaf0ee703ef619fb48c6e054f3ef246a916d79906adef09a1ad9254d7abc5"}
Jul 28 01:46:20.108552 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.108521    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:20.149296 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.149267    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd-maas-api-service-ca\") pod \"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd\" (UID: \"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd\") "
Jul 28 01:46:20.149668 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.149646    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c61b3d53-f3aa-44e8-b2d9-021e5e325ffd" (UID: "c61b3d53-f3aa-44e8-b2d9-021e5e325ffd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:20.249811 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.249729    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:20.979472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.979445    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"
Jul 28 01:46:20.979654 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.979488    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn" event={"ID":"c61b3d53-f3aa-44e8-b2d9-021e5e325ffd","Type":"ContainerDied","Data":"267f786d025938790a510bfe4d9272cf3150356a687a866756b536fe8d7cd6f4"}
Jul 28 01:46:20.979654 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:20.979539    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="267f786d025938790a510bfe4d9272cf3150356a687a866756b536fe8d7cd6f4"
Jul 28 01:46:32.885346 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:32.885314    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"]
Jul 28 01:46:32.885837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:32.885518    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8" podUID="b2a4da9c-4df1-45b5-9d77-538754ad9e7a" containerName="payload-processing" containerID="cri-o://897216f45929beacdb5e5b32a39a3e56f64fbc3ab93fd392acbbb5d32c8b759f" gracePeriod=30
Jul 28 01:46:32.898111 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:32.898081    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"]
Jul 28 01:46:32.898336 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:32.898289    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl" podUID="bbc49fa2-269e-4346-a3cd-0f7da71d6443" containerName="payload-pre-processing" containerID="cri-o://e34c55744ca2c3c34bfc5488518f1fc7f14ccf492ce87f42e05da2bc738e6090" gracePeriod=30
Jul 28 01:46:33.039566 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.039533    2566 generic.go:356] "Generic (PLEG): container finished" podID="b2a4da9c-4df1-45b5-9d77-538754ad9e7a" containerID="897216f45929beacdb5e5b32a39a3e56f64fbc3ab93fd392acbbb5d32c8b759f" exitCode=0
Jul 28 01:46:33.039736 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.039617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8" event={"ID":"b2a4da9c-4df1-45b5-9d77-538754ad9e7a","Type":"ContainerDied","Data":"897216f45929beacdb5e5b32a39a3e56f64fbc3ab93fd392acbbb5d32c8b759f"}
Jul 28 01:46:33.041311 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.041288    2566 generic.go:356] "Generic (PLEG): container finished" podID="bbc49fa2-269e-4346-a3cd-0f7da71d6443" containerID="e34c55744ca2c3c34bfc5488518f1fc7f14ccf492ce87f42e05da2bc738e6090" exitCode=0
Jul 28 01:46:33.041416 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.041370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl" event={"ID":"bbc49fa2-269e-4346-a3cd-0f7da71d6443","Type":"ContainerDied","Data":"e34c55744ca2c3c34bfc5488518f1fc7f14ccf492ce87f42e05da2bc738e6090"}
Jul 28 01:46:33.062223 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.062149    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"]
Jul 28 01:46:33.064716 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.064689    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-ea62b7-779bn"]
Jul 28 01:46:33.160294 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.160273    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:46:33.178682 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.178630    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:46:33.239308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.239266    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc49fa2-269e-4346-a3cd-0f7da71d6443-plugins-config-volume\") pod \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") "
Jul 28 01:46:33.239500 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.239347    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-plugins-config-volume\") pod \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") "
Jul 28 01:46:33.239500 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.239394    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbhs6\" (UniqueName: \"kubernetes.io/projected/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-kube-api-access-rbhs6\") pod \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\" (UID: \"b2a4da9c-4df1-45b5-9d77-538754ad9e7a\") "
Jul 28 01:46:33.239500 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.239429    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5p6w\" (UniqueName: \"kubernetes.io/projected/bbc49fa2-269e-4346-a3cd-0f7da71d6443-kube-api-access-l5p6w\") pod \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\" (UID: \"bbc49fa2-269e-4346-a3cd-0f7da71d6443\") "
Jul 28 01:46:33.239754 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.239725    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc49fa2-269e-4346-a3cd-0f7da71d6443-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bbc49fa2-269e-4346-a3cd-0f7da71d6443" (UID: "bbc49fa2-269e-4346-a3cd-0f7da71d6443"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:33.239802 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.239771    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2a4da9c-4df1-45b5-9d77-538754ad9e7a" (UID: "b2a4da9c-4df1-45b5-9d77-538754ad9e7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:33.241716 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.241689    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc49fa2-269e-4346-a3cd-0f7da71d6443-kube-api-access-l5p6w" (OuterVolumeSpecName: "kube-api-access-l5p6w") pod "bbc49fa2-269e-4346-a3cd-0f7da71d6443" (UID: "bbc49fa2-269e-4346-a3cd-0f7da71d6443"). InnerVolumeSpecName "kube-api-access-l5p6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:46:33.241798 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.241762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-kube-api-access-rbhs6" (OuterVolumeSpecName: "kube-api-access-rbhs6") pod "b2a4da9c-4df1-45b5-9d77-538754ad9e7a" (UID: "b2a4da9c-4df1-45b5-9d77-538754ad9e7a"). InnerVolumeSpecName "kube-api-access-rbhs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:46:33.340146 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.340100    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bbc49fa2-269e-4346-a3cd-0f7da71d6443-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:33.340146 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.340144    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:33.340146 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.340157    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbhs6\" (UniqueName: \"kubernetes.io/projected/b2a4da9c-4df1-45b5-9d77-538754ad9e7a-kube-api-access-rbhs6\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:33.340410 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:33.340171    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5p6w\" (UniqueName: \"kubernetes.io/projected/bbc49fa2-269e-4346-a3cd-0f7da71d6443-kube-api-access-l5p6w\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:34.047607 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.047575    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"
Jul 28 01:46:34.048088 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.047621    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl" event={"ID":"bbc49fa2-269e-4346-a3cd-0f7da71d6443","Type":"ContainerDied","Data":"5dda159f286a39a72b6afe7aa32424b82be9b71b075d3ab0c4d9e532a87b3090"}
Jul 28 01:46:34.048088 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.047681    2566 scope.go:117] "RemoveContainer" containerID="e34c55744ca2c3c34bfc5488518f1fc7f14ccf492ce87f42e05da2bc738e6090"
Jul 28 01:46:34.049070 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.049045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8" event={"ID":"b2a4da9c-4df1-45b5-9d77-538754ad9e7a","Type":"ContainerDied","Data":"4bf9ac80468045cf5fe90b0e3199cfa8da07774297fd19f1558a0b20cf57da8c"}
Jul 28 01:46:34.049125 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.049082    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"
Jul 28 01:46:34.057851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.057724    2566 scope.go:117] "RemoveContainer" containerID="897216f45929beacdb5e5b32a39a3e56f64fbc3ab93fd392acbbb5d32c8b759f"
Jul 28 01:46:34.078199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.077936    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"]
Jul 28 01:46:34.079796 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.079773    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ea62b7-7b7cdf6664-9k5pl"]
Jul 28 01:46:34.086596 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.086569    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"]
Jul 28 01:46:34.091225 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.091193    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ea62b7-7cb4d68766-6mzd8"]
Jul 28 01:46:34.562223 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.562193    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2a4da9c-4df1-45b5-9d77-538754ad9e7a" path="/var/lib/kubelet/pods/b2a4da9c-4df1-45b5-9d77-538754ad9e7a/volumes"
Jul 28 01:46:34.562517 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.562505    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbc49fa2-269e-4346-a3cd-0f7da71d6443" path="/var/lib/kubelet/pods/bbc49fa2-269e-4346-a3cd-0f7da71d6443/volumes"
Jul 28 01:46:34.562774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:34.562764    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c61b3d53-f3aa-44e8-b2d9-021e5e325ffd" path="/var/lib/kubelet/pods/c61b3d53-f3aa-44e8-b2d9-021e5e325ffd/volumes"
Jul 28 01:46:35.224710 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:35.224674    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:46:39.645302 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645254    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"]
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645585    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c61b3d53-f3aa-44e8-b2d9-021e5e325ffd" containerName="revoke-keys"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645597    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c61b3d53-f3aa-44e8-b2d9-021e5e325ffd" containerName="revoke-keys"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645612    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a4da9c-4df1-45b5-9d77-538754ad9e7a" containerName="payload-processing"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645618    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a4da9c-4df1-45b5-9d77-538754ad9e7a" containerName="payload-processing"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645625    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbc49fa2-269e-4346-a3cd-0f7da71d6443" containerName="payload-pre-processing"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645631    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbc49fa2-269e-4346-a3cd-0f7da71d6443" containerName="payload-pre-processing"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645680    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2a4da9c-4df1-45b5-9d77-538754ad9e7a" containerName="payload-processing"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645689    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c61b3d53-f3aa-44e8-b2d9-021e5e325ffd" containerName="revoke-keys"
Jul 28 01:46:39.645696 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.645696    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbc49fa2-269e-4346-a3cd-0f7da71d6443" containerName="payload-pre-processing"
Jul 28 01:46:39.655024 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.654933    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:39.655024 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.655004    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"]
Jul 28 01:46:39.657332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.657311    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:46:39.657480 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.657377    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:46:39.687409 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.687369    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19881970-35a1-4214-8c31-de31e6e4b783-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj\" (UID: \"19881970-35a1-4214-8c31-de31e6e4b783\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:39.787887 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.787833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19881970-35a1-4214-8c31-de31e6e4b783-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj\" (UID: \"19881970-35a1-4214-8c31-de31e6e4b783\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:39.788502 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.788481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19881970-35a1-4214-8c31-de31e6e4b783-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj\" (UID: \"19881970-35a1-4214-8c31-de31e6e4b783\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:39.966296 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:39.966214    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:40.108093 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:40.108043    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"]
Jul 28 01:46:40.110272 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:46:40.110241    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19881970_35a1_4214_8c31_de31e6e4b783.slice/crio-0de4bfd85a6d8afbcc092981c9f160a378bd031eeae8cf7c2825fdebc6292dac WatchSource:0}: Error finding container 0de4bfd85a6d8afbcc092981c9f160a378bd031eeae8cf7c2825fdebc6292dac: Status 404 returned error can't find the container with id 0de4bfd85a6d8afbcc092981c9f160a378bd031eeae8cf7c2825fdebc6292dac
Jul 28 01:46:41.088589 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:41.088546    2566 generic.go:356] "Generic (PLEG): container finished" podID="19881970-35a1-4214-8c31-de31e6e4b783" containerID="5d5fbf98934f4729de28c4702711a61e108e5d21f9a762270b0de5720ca79a9e" exitCode=0
Jul 28 01:46:41.089038 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:41.088640    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj" event={"ID":"19881970-35a1-4214-8c31-de31e6e4b783","Type":"ContainerDied","Data":"5d5fbf98934f4729de28c4702711a61e108e5d21f9a762270b0de5720ca79a9e"}
Jul 28 01:46:41.089038 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:41.088702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj" event={"ID":"19881970-35a1-4214-8c31-de31e6e4b783","Type":"ContainerStarted","Data":"0de4bfd85a6d8afbcc092981c9f160a378bd031eeae8cf7c2825fdebc6292dac"}
Jul 28 01:46:42.228446 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:42.228415    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:42.305311 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:42.305278    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19881970-35a1-4214-8c31-de31e6e4b783-maas-api-service-ca\") pod \"19881970-35a1-4214-8c31-de31e6e4b783\" (UID: \"19881970-35a1-4214-8c31-de31e6e4b783\") "
Jul 28 01:46:42.305639 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:42.305617    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19881970-35a1-4214-8c31-de31e6e4b783-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19881970-35a1-4214-8c31-de31e6e4b783" (UID: "19881970-35a1-4214-8c31-de31e6e4b783"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:42.406362 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:42.406278    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19881970-35a1-4214-8c31-de31e6e4b783-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:43.098428 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:43.098397    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"
Jul 28 01:46:43.098428 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:43.098414    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj" event={"ID":"19881970-35a1-4214-8c31-de31e6e4b783","Type":"ContainerDied","Data":"0de4bfd85a6d8afbcc092981c9f160a378bd031eeae8cf7c2825fdebc6292dac"}
Jul 28 01:46:43.098642 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:43.098457    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0de4bfd85a6d8afbcc092981c9f160a378bd031eeae8cf7c2825fdebc6292dac"
Jul 28 01:46:55.342599 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.342563    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"]
Jul 28 01:46:55.343326 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.342854    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb" podUID="acd6f766-b8d0-4f1b-a52b-95f8eee6deff" containerName="payload-processing" containerID="cri-o://17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585" gracePeriod=30
Jul 28 01:46:55.350382 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.350025    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"]
Jul 28 01:46:55.350382 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.350258    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg" podUID="957056c4-613e-4a20-ace4-4fb0e3e50566" containerName="payload-pre-processing" containerID="cri-o://d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e" gracePeriod=30
Jul 28 01:46:55.503585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.503520    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"]
Jul 28 01:46:55.505626 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.505595    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-5e7618-2jxjj"]
Jul 28 01:46:55.623785 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.623757    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:46:55.630522 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.630493    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:46:55.710730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.710695    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzsgk\" (UniqueName: \"kubernetes.io/projected/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-kube-api-access-jzsgk\") pod \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") "
Jul 28 01:46:55.710730 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.710738    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957056c4-613e-4a20-ace4-4fb0e3e50566-plugins-config-volume\") pod \"957056c4-613e-4a20-ace4-4fb0e3e50566\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") "
Jul 28 01:46:55.711015 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.710773    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fmkm\" (UniqueName: \"kubernetes.io/projected/957056c4-613e-4a20-ace4-4fb0e3e50566-kube-api-access-7fmkm\") pod \"957056c4-613e-4a20-ace4-4fb0e3e50566\" (UID: \"957056c4-613e-4a20-ace4-4fb0e3e50566\") "
Jul 28 01:46:55.711015 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.710822    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-plugins-config-volume\") pod \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\" (UID: \"acd6f766-b8d0-4f1b-a52b-95f8eee6deff\") "
Jul 28 01:46:55.711226 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.711187    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957056c4-613e-4a20-ace4-4fb0e3e50566-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "957056c4-613e-4a20-ace4-4fb0e3e50566" (UID: "957056c4-613e-4a20-ace4-4fb0e3e50566"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:55.711362 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.711235    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "acd6f766-b8d0-4f1b-a52b-95f8eee6deff" (UID: "acd6f766-b8d0-4f1b-a52b-95f8eee6deff"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:46:55.713121 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.713099    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-kube-api-access-jzsgk" (OuterVolumeSpecName: "kube-api-access-jzsgk") pod "acd6f766-b8d0-4f1b-a52b-95f8eee6deff" (UID: "acd6f766-b8d0-4f1b-a52b-95f8eee6deff"). InnerVolumeSpecName "kube-api-access-jzsgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:46:55.713218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.713202    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957056c4-613e-4a20-ace4-4fb0e3e50566-kube-api-access-7fmkm" (OuterVolumeSpecName: "kube-api-access-7fmkm") pod "957056c4-613e-4a20-ace4-4fb0e3e50566" (UID: "957056c4-613e-4a20-ace4-4fb0e3e50566"). InnerVolumeSpecName "kube-api-access-7fmkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:46:55.811899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.811860    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/957056c4-613e-4a20-ace4-4fb0e3e50566-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:55.811899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.811893    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fmkm\" (UniqueName: \"kubernetes.io/projected/957056c4-613e-4a20-ace4-4fb0e3e50566-kube-api-access-7fmkm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:55.811899 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.811904    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:55.812179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:55.811913    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzsgk\" (UniqueName: \"kubernetes.io/projected/acd6f766-b8d0-4f1b-a52b-95f8eee6deff-kube-api-access-jzsgk\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:46:56.161924 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.161890    2566 generic.go:356] "Generic (PLEG): container finished" podID="acd6f766-b8d0-4f1b-a52b-95f8eee6deff" containerID="17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585" exitCode=0
Jul 28 01:46:56.162144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.161931    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"
Jul 28 01:46:56.162144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.161999    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb" event={"ID":"acd6f766-b8d0-4f1b-a52b-95f8eee6deff","Type":"ContainerDied","Data":"17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585"}
Jul 28 01:46:56.162144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.162059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb" event={"ID":"acd6f766-b8d0-4f1b-a52b-95f8eee6deff","Type":"ContainerDied","Data":"4c5fe9ed70cb63dfd81654e54d0db45f6c499c0a301f8d12b84d8371a5a568e8"}
Jul 28 01:46:56.162144 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.162090    2566 scope.go:117] "RemoveContainer" containerID="17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585"
Jul 28 01:46:56.163430 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.163410    2566 generic.go:356] "Generic (PLEG): container finished" podID="957056c4-613e-4a20-ace4-4fb0e3e50566" containerID="d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e" exitCode=0
Jul 28 01:46:56.163508 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.163462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg" event={"ID":"957056c4-613e-4a20-ace4-4fb0e3e50566","Type":"ContainerDied","Data":"d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e"}
Jul 28 01:46:56.163508 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.163493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg" event={"ID":"957056c4-613e-4a20-ace4-4fb0e3e50566","Type":"ContainerDied","Data":"271a6ce988d160f7ed24fecda3031d1ab5346ee857e3a1b48d938f3dfbd7556c"}
Jul 28 01:46:56.163508 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.163467    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"
Jul 28 01:46:56.174647 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.174623    2566 scope.go:117] "RemoveContainer" containerID="17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585"
Jul 28 01:46:56.175024 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:46:56.174905    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585\": container with ID starting with 17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585 not found: ID does not exist" containerID="17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585"
Jul 28 01:46:56.175024 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.174935    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585"} err="failed to get container status \"17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585\": rpc error: code = NotFound desc = could not find container \"17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585\": container with ID starting with 17e9818fecaf294411d42773c4cdca6a1dcfc9748cb768cc56984e6ac60ca585 not found: ID does not exist"
Jul 28 01:46:56.175024 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.174961    2566 scope.go:117] "RemoveContainer" containerID="d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e"
Jul 28 01:46:56.182531 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.182514    2566 scope.go:117] "RemoveContainer" containerID="d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e"
Jul 28 01:46:56.182756 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:46:56.182741    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e\": container with ID starting with d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e not found: ID does not exist" containerID="d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e"
Jul 28 01:46:56.182803 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.182762    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e"} err="failed to get container status \"d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e\": rpc error: code = NotFound desc = could not find container \"d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e\": container with ID starting with d82d52b6f7788465cc1c09f78e4628a40b5b945b0573b02ffe5cb35f0247aa2e not found: ID does not exist"
Jul 28 01:46:56.188192 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.188169    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"]
Jul 28 01:46:56.194568 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.194540    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-5e7618-7cfc88cdfd-ngwfb"]
Jul 28 01:46:56.202418 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.202397    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"]
Jul 28 01:46:56.206509 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.206490    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-5e7618-564896d5dd-sz6jg"]
Jul 28 01:46:56.560716 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.560636    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19881970-35a1-4214-8c31-de31e6e4b783" path="/var/lib/kubelet/pods/19881970-35a1-4214-8c31-de31e6e4b783/volumes"
Jul 28 01:46:56.561076 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.560956    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="957056c4-613e-4a20-ace4-4fb0e3e50566" path="/var/lib/kubelet/pods/957056c4-613e-4a20-ace4-4fb0e3e50566/volumes"
Jul 28 01:46:56.561277 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:46:56.561264    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd6f766-b8d0-4f1b-a52b-95f8eee6deff" path="/var/lib/kubelet/pods/acd6f766-b8d0-4f1b-a52b-95f8eee6deff/volumes"
Jul 28 01:47:00.236567 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:00.236524    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:47:06.612115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612080    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"]
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612413    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19881970-35a1-4214-8c31-de31e6e4b783" containerName="revoke-keys"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612424    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="19881970-35a1-4214-8c31-de31e6e4b783" containerName="revoke-keys"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612456    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="957056c4-613e-4a20-ace4-4fb0e3e50566" containerName="payload-pre-processing"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612461    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="957056c4-613e-4a20-ace4-4fb0e3e50566" containerName="payload-pre-processing"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612472    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd6f766-b8d0-4f1b-a52b-95f8eee6deff" containerName="payload-processing"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612478    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd6f766-b8d0-4f1b-a52b-95f8eee6deff" containerName="payload-processing"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612528    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd6f766-b8d0-4f1b-a52b-95f8eee6deff" containerName="payload-processing"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612534    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="19881970-35a1-4214-8c31-de31e6e4b783" containerName="revoke-keys"
Jul 28 01:47:06.612619 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.612540    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="957056c4-613e-4a20-ace4-4fb0e3e50566" containerName="payload-pre-processing"
Jul 28 01:47:06.615473 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.615457    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.622238 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.622212    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-0337e8\""
Jul 28 01:47:06.634343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.634315    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"]
Jul 28 01:47:06.658512 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.658481    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"]
Jul 28 01:47:06.661795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.661778    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.665421 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.665400    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-0337e8-dockercfg-xt5lv\""
Jul 28 01:47:06.677621 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.677597    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"]
Jul 28 01:47:06.701684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.701648    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f720199-3725-4252-8280-a5122beb7590-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.701863 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.701735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfg5h\" (UniqueName: \"kubernetes.io/projected/2f720199-3725-4252-8280-a5122beb7590-kube-api-access-hfg5h\") pod \"payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.802931 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.802890    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f720199-3725-4252-8280-a5122beb7590-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.803137 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.802952    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hdn6\" (UniqueName: \"kubernetes.io/projected/cef226bd-348d-40a3-88f9-b8a124df15b1-kube-api-access-9hdn6\") pod \"payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.803137 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.803024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfg5h\" (UniqueName: \"kubernetes.io/projected/2f720199-3725-4252-8280-a5122beb7590-kube-api-access-hfg5h\") pod \"payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.803137 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.803051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cef226bd-348d-40a3-88f9-b8a124df15b1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.803743 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.803721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f720199-3725-4252-8280-a5122beb7590-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.812924 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.812897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfg5h\" (UniqueName: \"kubernetes.io/projected/2f720199-3725-4252-8280-a5122beb7590-kube-api-access-hfg5h\") pod \"payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.904561 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.904461    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cef226bd-348d-40a3-88f9-b8a124df15b1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.904714 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.904617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hdn6\" (UniqueName: \"kubernetes.io/projected/cef226bd-348d-40a3-88f9-b8a124df15b1-kube-api-access-9hdn6\") pod \"payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.905296 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.905277    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cef226bd-348d-40a3-88f9-b8a124df15b1-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.912765 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.912741    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hdn6\" (UniqueName: \"kubernetes.io/projected/cef226bd-348d-40a3-88f9-b8a124df15b1-kube-api-access-9hdn6\") pod \"payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") " pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:06.925609 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.925581    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:06.972066 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:06.972030    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:07.079208 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.079174    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"]
Jul 28 01:47:07.083167 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:47:07.083132    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f720199_3725_4252_8280_a5122beb7590.slice/crio-d9809c1ed21e9be2cdac0fd2360c10fcbf75c119f64484f9e9982b8b0f62f975 WatchSource:0}: Error finding container d9809c1ed21e9be2cdac0fd2360c10fcbf75c119f64484f9e9982b8b0f62f975: Status 404 returned error can't find the container with id d9809c1ed21e9be2cdac0fd2360c10fcbf75c119f64484f9e9982b8b0f62f975
Jul 28 01:47:07.125653 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.125619    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"]
Jul 28 01:47:07.136218 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:47:07.136189    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef226bd_348d_40a3_88f9_b8a124df15b1.slice/crio-316e89b0f1969fac74fe9659c0e9643ff7e39b958fee387c687f02a38ead0a62 WatchSource:0}: Error finding container 316e89b0f1969fac74fe9659c0e9643ff7e39b958fee387c687f02a38ead0a62: Status 404 returned error can't find the container with id 316e89b0f1969fac74fe9659c0e9643ff7e39b958fee387c687f02a38ead0a62
Jul 28 01:47:07.224956 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.224923    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9" event={"ID":"cef226bd-348d-40a3-88f9-b8a124df15b1","Type":"ContainerStarted","Data":"37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b"}
Jul 28 01:47:07.225186 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.225125    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:07.225266 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.225186    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9" event={"ID":"cef226bd-348d-40a3-88f9-b8a124df15b1","Type":"ContainerStarted","Data":"316e89b0f1969fac74fe9659c0e9643ff7e39b958fee387c687f02a38ead0a62"}
Jul 28 01:47:07.226566 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.226543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6" event={"ID":"2f720199-3725-4252-8280-a5122beb7590","Type":"ContainerStarted","Data":"a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93"}
Jul 28 01:47:07.226704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.226578    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6" event={"ID":"2f720199-3725-4252-8280-a5122beb7590","Type":"ContainerStarted","Data":"d9809c1ed21e9be2cdac0fd2360c10fcbf75c119f64484f9e9982b8b0f62f975"}
Jul 28 01:47:07.226704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.226679    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:07.243419 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.243351    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9" podStartSLOduration=1.243331997 podStartE2EDuration="1.243331997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:07.241242704 +0000 UTC m=+2563.252481315" watchObservedRunningTime="2026-07-28 01:47:07.243331997 +0000 UTC m=+2563.254570607"
Jul 28 01:47:07.258960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:07.258875    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6" podStartSLOduration=1.258855214 podStartE2EDuration="1.258855214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:07.258146259 +0000 UTC m=+2563.269384869" watchObservedRunningTime="2026-07-28 01:47:07.258855214 +0000 UTC m=+2563.270093824"
Jul 28 01:47:11.233120 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:11.233083    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:47:18.233947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:18.233914    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 01:47:18.234343 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:18.234020    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 01:47:24.804506 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:24.804470    2566 scope.go:117] "RemoveContainer" containerID="cdc6a9d15e4724cec01ad6d60c31de4b5c1350c5175ed9b100cb6bcf0f6aa68a"
Jul 28 01:47:35.100069 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.100032    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"]
Jul 28 01:47:35.103445 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.103424    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.106023 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.105999    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-44b320\""
Jul 28 01:47:35.117657 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.117621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"]
Jul 28 01:47:35.123646 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.123618    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"]
Jul 28 01:47:35.127180 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.127159    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.129805 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.129784    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-44b320-dockercfg-vksgk\""
Jul 28 01:47:35.139832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.139799    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"]
Jul 28 01:47:35.253674 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.253631    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hngp\" (UniqueName: \"kubernetes.io/projected/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-kube-api-access-4hngp\") pod \"payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") " pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.253871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.253687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") " pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.253871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.253760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gcsk\" (UniqueName: \"kubernetes.io/projected/261c1507-f81a-47bd-b03f-98ac08fbf5e4-kube-api-access-9gcsk\") pod \"payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.253871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.253813    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c1507-f81a-47bd-b03f-98ac08fbf5e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.354467 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.354375    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hngp\" (UniqueName: \"kubernetes.io/projected/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-kube-api-access-4hngp\") pod \"payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") " pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.354467 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.354428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") " pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.354467 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.354459    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gcsk\" (UniqueName: \"kubernetes.io/projected/261c1507-f81a-47bd-b03f-98ac08fbf5e4-kube-api-access-9gcsk\") pod \"payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.354692 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.354483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c1507-f81a-47bd-b03f-98ac08fbf5e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.355252 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.355232    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c1507-f81a-47bd-b03f-98ac08fbf5e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.355307 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.355279    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") " pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.362750 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.362729    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gcsk\" (UniqueName: \"kubernetes.io/projected/261c1507-f81a-47bd-b03f-98ac08fbf5e4-kube-api-access-9gcsk\") pod \"payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.362861 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.362844    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hngp\" (UniqueName: \"kubernetes.io/projected/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-kube-api-access-4hngp\") pod \"payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") " pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.414490 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.414445    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:35.439811 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.439772    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:35.572193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.572162    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"]
Jul 28 01:47:35.574376 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:47:35.574346    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261c1507_f81a_47bd_b03f_98ac08fbf5e4.slice/crio-1b2865e0acde47a8c09e3af0ac14eb8330ea16d852dac9ff222db51734ca31af WatchSource:0}: Error finding container 1b2865e0acde47a8c09e3af0ac14eb8330ea16d852dac9ff222db51734ca31af: Status 404 returned error can't find the container with id 1b2865e0acde47a8c09e3af0ac14eb8330ea16d852dac9ff222db51734ca31af
Jul 28 01:47:35.597132 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:35.597108    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"]
Jul 28 01:47:35.606086 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:47:35.606055    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f550aa8_c619_4dd0_ba2d_cd75e2d8fa64.slice/crio-0d9f82b4c127493a051c9dab696249c1506cb056d9df293a27f5670c5b1cfdba WatchSource:0}: Error finding container 0d9f82b4c127493a051c9dab696249c1506cb056d9df293a27f5670c5b1cfdba: Status 404 returned error can't find the container with id 0d9f82b4c127493a051c9dab696249c1506cb056d9df293a27f5670c5b1cfdba
Jul 28 01:47:36.384934 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.384884    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt" event={"ID":"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64","Type":"ContainerStarted","Data":"256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8"}
Jul 28 01:47:36.385444 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.384953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt" event={"ID":"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64","Type":"ContainerStarted","Data":"0d9f82b4c127493a051c9dab696249c1506cb056d9df293a27f5670c5b1cfdba"}
Jul 28 01:47:36.385444 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.384995    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:47:36.386649 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.386629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw" event={"ID":"261c1507-f81a-47bd-b03f-98ac08fbf5e4","Type":"ContainerStarted","Data":"819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a"}
Jul 28 01:47:36.386802 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.386669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw" event={"ID":"261c1507-f81a-47bd-b03f-98ac08fbf5e4","Type":"ContainerStarted","Data":"1b2865e0acde47a8c09e3af0ac14eb8330ea16d852dac9ff222db51734ca31af"}
Jul 28 01:47:36.386802 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.386716    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:36.401049 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.401003    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt" podStartSLOduration=1.400987425 podStartE2EDuration="1.400987425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:36.400064384 +0000 UTC m=+2592.411302994" watchObservedRunningTime="2026-07-28 01:47:36.400987425 +0000 UTC m=+2592.412226032"
Jul 28 01:47:36.415810 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:36.415757    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw" podStartSLOduration=1.41574233 podStartE2EDuration="1.41574233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:47:36.41406591 +0000 UTC m=+2592.425304532" watchObservedRunningTime="2026-07-28 01:47:36.41574233 +0000 UTC m=+2592.426980938"
Jul 28 01:47:41.243404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:41.243371    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:47:47.393088 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:47.393000    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 01:47:47.393541 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:47:47.393184    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 01:48:05.415415 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.415380    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"]
Jul 28 01:48:05.418804 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.418782    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.424739 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.424714    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"default-dockercfg-z24k8\""
Jul 28 01:48:05.424987 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.424950    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"kube-root-ca.crt\""
Jul 28 01:48:05.425091 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.425029    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"openshift-service-ca.crt\""
Jul 28 01:48:05.425091 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.425077    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"auth-test-model-0337e8-kserve-self-signed-certs\""
Jul 28 01:48:05.431605 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.431579    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"]
Jul 28 01:48:05.518243 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.518213    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-tmp-dir\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.518243 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.518248    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-model-cache\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.518535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.518271    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-dshm\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.518535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.518287    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsb67\" (UniqueName: \"kubernetes.io/projected/c2562858-62a1-4b31-9518-cf955c5df1f1-kube-api-access-zsb67\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.518535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.518449    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-home\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.518535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.518493    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2562858-62a1-4b31-9518-cf955c5df1f1-tls-certs\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619209    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-home\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619247 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2562858-62a1-4b31-9518-cf955c5df1f1-tls-certs\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619502 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-tmp-dir\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619502 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-model-cache\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619502 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-dshm\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619502 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619450    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsb67\" (UniqueName: \"kubernetes.io/projected/c2562858-62a1-4b31-9518-cf955c5df1f1-kube-api-access-zsb67\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619706 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619664    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-home\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619767 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619723    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-tmp-dir\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.619821 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.619774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-model-cache\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.621814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.621797    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-dshm\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.622119 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.622104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2562858-62a1-4b31-9518-cf955c5df1f1-tls-certs\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.629582 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.629557    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsb67\" (UniqueName: \"kubernetes.io/projected/c2562858-62a1-4b31-9518-cf955c5df1f1-kube-api-access-zsb67\") pod \"auth-test-model-0337e8-kserve-589868579-745z8\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") " pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.728816 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.728722    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:05.891845 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:05.891816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"]
Jul 28 01:48:05.894770 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:48:05.894741    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2562858_62a1_4b31_9518_cf955c5df1f1.slice/crio-0bdc284a8da47ed97380d00be9e4dfa32a73b3a1c584a8a1599be74a385e21c5 WatchSource:0}: Error finding container 0bdc284a8da47ed97380d00be9e4dfa32a73b3a1c584a8a1599be74a385e21c5: Status 404 returned error can't find the container with id 0bdc284a8da47ed97380d00be9e4dfa32a73b3a1c584a8a1599be74a385e21c5
Jul 28 01:48:06.536593 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:06.536557    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8" event={"ID":"c2562858-62a1-4b31-9518-cf955c5df1f1","Type":"ContainerStarted","Data":"4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804"}
Jul 28 01:48:06.537021 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:06.536605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8" event={"ID":"c2562858-62a1-4b31-9518-cf955c5df1f1","Type":"ContainerStarted","Data":"0bdc284a8da47ed97380d00be9e4dfa32a73b3a1c584a8a1599be74a385e21c5"}
Jul 28 01:48:06.556381 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:06.556329    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8" podStartSLOduration=1.556313094 podStartE2EDuration="1.556313094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:48:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:48:06.554772732 +0000 UTC m=+2622.566011342" watchObservedRunningTime="2026-07-28 01:48:06.556313094 +0000 UTC m=+2622.567551703"
Jul 28 01:48:08.334800 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:08.334761    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:48:15.729736 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:15.729694    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:15.730290 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:15.729766    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:15.742538 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:15.742512    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:16.595897 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:16.595862    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:48:29.333585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:29.333546    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:48:54.940365 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:48:54.940327    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:49:16.133267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:49:16.133229    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:49:29.534514 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:49:29.534483    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:49:41.833355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:49:41.833320    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:49:59.533147 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:49:59.533110    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:50:17.727005 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:50:17.726954    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:50:44.239569 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:50:44.239487    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:50:59.938576 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:50:59.938535    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:51:14.138804 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:51:14.138765    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:51:24.860292 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:51:24.860257    2566 scope.go:117] "RemoveContainer" containerID="b931199fdb3d139f34cd3c780de3621537f5a7f9a932eed79151491af4729237"
Jul 28 01:51:24.868851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:51:24.868832    2566 scope.go:117] "RemoveContainer" containerID="5295272efabb2b24eb46c13ec0ca17460bb4eeaba65b6c8cf68f9b3d90f3dd1b"
Jul 28 01:51:24.876296 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:51:24.876276    2566 scope.go:117] "RemoveContainer" containerID="4fd29be374f8007f491c5ac53efdafe640a537a46fa7284ddfb08ead5b28b111"
Jul 28 01:51:35.735825 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:51:35.735783    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:51:48.532261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:51:48.532222    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:52:09.137397 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:52:09.137362    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:52:24.900922 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:52:24.900879    2566 scope.go:117] "RemoveContainer" containerID="a5faaf0ee703ef619fb48c6e054f3ef246a916d79906adef09a1ad9254d7abc5"
Jul 28 01:52:26.536907 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:52:26.536869    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:52:46.642961 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:52:46.642925    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:52:52.198082 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:52:52.198046    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"]
Jul 28 01:53:03.827262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:03.827228    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:53:07.217706 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.217643    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8" podUID="c2562858-62a1-4b31-9518-cf955c5df1f1" containerName="main" containerID="cri-o://4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804" gracePeriod=45
Jul 28 01:53:07.467289 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.467265    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:53:07.640996 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.640949    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-model-cache\") pod \"c2562858-62a1-4b31-9518-cf955c5df1f1\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") "
Jul 28 01:53:07.641199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641003    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-home\") pod \"c2562858-62a1-4b31-9518-cf955c5df1f1\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") "
Jul 28 01:53:07.641199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641030    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2562858-62a1-4b31-9518-cf955c5df1f1-tls-certs\") pod \"c2562858-62a1-4b31-9518-cf955c5df1f1\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") "
Jul 28 01:53:07.641199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641118    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-dshm\") pod \"c2562858-62a1-4b31-9518-cf955c5df1f1\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") "
Jul 28 01:53:07.641199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641163    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsb67\" (UniqueName: \"kubernetes.io/projected/c2562858-62a1-4b31-9518-cf955c5df1f1-kube-api-access-zsb67\") pod \"c2562858-62a1-4b31-9518-cf955c5df1f1\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") "
Jul 28 01:53:07.641199 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641190    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-tmp-dir\") pod \"c2562858-62a1-4b31-9518-cf955c5df1f1\" (UID: \"c2562858-62a1-4b31-9518-cf955c5df1f1\") "
Jul 28 01:53:07.641485 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641229    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "c2562858-62a1-4b31-9518-cf955c5df1f1" (UID: "c2562858-62a1-4b31-9518-cf955c5df1f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:07.641485 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641311    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-home" (OuterVolumeSpecName: "home") pod "c2562858-62a1-4b31-9518-cf955c5df1f1" (UID: "c2562858-62a1-4b31-9518-cf955c5df1f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:07.641602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641534    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:53:07.641602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641552    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:53:07.641602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.641587    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c2562858-62a1-4b31-9518-cf955c5df1f1" (UID: "c2562858-62a1-4b31-9518-cf955c5df1f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:07.643673 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.643643    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2562858-62a1-4b31-9518-cf955c5df1f1-kube-api-access-zsb67" (OuterVolumeSpecName: "kube-api-access-zsb67") pod "c2562858-62a1-4b31-9518-cf955c5df1f1" (UID: "c2562858-62a1-4b31-9518-cf955c5df1f1"). InnerVolumeSpecName "kube-api-access-zsb67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:53:07.643673 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.643653    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2562858-62a1-4b31-9518-cf955c5df1f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c2562858-62a1-4b31-9518-cf955c5df1f1" (UID: "c2562858-62a1-4b31-9518-cf955c5df1f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:53:07.643812 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.643674    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-dshm" (OuterVolumeSpecName: "dshm") pod "c2562858-62a1-4b31-9518-cf955c5df1f1" (UID: "c2562858-62a1-4b31-9518-cf955c5df1f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:53:07.742879 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.742839    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:53:07.742879 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.742874    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsb67\" (UniqueName: \"kubernetes.io/projected/c2562858-62a1-4b31-9518-cf955c5df1f1-kube-api-access-zsb67\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:53:07.742879 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.742886    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2562858-62a1-4b31-9518-cf955c5df1f1-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:53:07.743161 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:07.742896    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2562858-62a1-4b31-9518-cf955c5df1f1-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:53:08.029398 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.029298    2566 generic.go:356] "Generic (PLEG): container finished" podID="c2562858-62a1-4b31-9518-cf955c5df1f1" containerID="4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804" exitCode=0
Jul 28 01:53:08.029398 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.029344    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"
Jul 28 01:53:08.029583 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.029388    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8" event={"ID":"c2562858-62a1-4b31-9518-cf955c5df1f1","Type":"ContainerDied","Data":"4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804"}
Jul 28 01:53:08.029583 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.029445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8" event={"ID":"c2562858-62a1-4b31-9518-cf955c5df1f1","Type":"ContainerDied","Data":"0bdc284a8da47ed97380d00be9e4dfa32a73b3a1c584a8a1599be74a385e21c5"}
Jul 28 01:53:08.029583 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.029470    2566 scope.go:117] "RemoveContainer" containerID="4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804"
Jul 28 01:53:08.039541 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.039519    2566 scope.go:117] "RemoveContainer" containerID="4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804"
Jul 28 01:53:08.039838 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:53:08.039816    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804\": container with ID starting with 4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804 not found: ID does not exist" containerID="4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804"
Jul 28 01:53:08.039928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.039847    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804"} err="failed to get container status \"4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804\": rpc error: code = NotFound desc = could not find container \"4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804\": container with ID starting with 4dfa99c6de82a48345c86accc897a0735954ac268245a155798987ebdb92b804 not found: ID does not exist"
Jul 28 01:53:08.050719 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.050692    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"]
Jul 28 01:53:08.053777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.053756    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/auth-test-model-0337e8-kserve-589868579-745z8"]
Jul 28 01:53:08.561359 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:08.561326    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2562858-62a1-4b31-9518-cf955c5df1f1" path="/var/lib/kubelet/pods/c2562858-62a1-4b31-9518-cf955c5df1f1/volumes"
Jul 28 01:53:11.620484 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.620446    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"]
Jul 28 01:53:11.620850 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.620792    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2562858-62a1-4b31-9518-cf955c5df1f1" containerName="main"
Jul 28 01:53:11.620850 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.620803    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2562858-62a1-4b31-9518-cf955c5df1f1" containerName="main"
Jul 28 01:53:11.620926 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.620862    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2562858-62a1-4b31-9518-cf955c5df1f1" containerName="main"
Jul 28 01:53:11.625617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.625595    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.629173 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.629076    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"sub-test-model-0337e8-kserve-self-signed-certs\""
Jul 28 01:53:11.629173 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.629102    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"openshift-service-ca.crt\""
Jul 28 01:53:11.629173 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.629168    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"kube-root-ca.crt\""
Jul 28 01:53:11.629394 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.629084    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"default-dockercfg-z24k8\""
Jul 28 01:53:11.635086 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.635062    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"]
Jul 28 01:53:11.776110 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.776070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-dshm\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.776110 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.776110    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-model-cache\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.776394 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.776133    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-home\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.776394 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.776286    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwg9r\" (UniqueName: \"kubernetes.io/projected/a6e7850b-340a-4927-9066-f93a01337fed-kube-api-access-zwg9r\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.776394 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.776325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6e7850b-340a-4927-9066-f93a01337fed-tls-certs\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.776514 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.776404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-tmp-dir\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877488 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-dshm\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877488 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-model-cache\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877488 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-home\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877796 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwg9r\" (UniqueName: \"kubernetes.io/projected/a6e7850b-340a-4927-9066-f93a01337fed-kube-api-access-zwg9r\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877796 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6e7850b-340a-4927-9066-f93a01337fed-tls-certs\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877796 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877534    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-tmp-dir\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877915    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-home\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.877960 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877933    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-model-cache\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.878091 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.877988    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-tmp-dir\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.879846 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.879824    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-dshm\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.880122 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.880103    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6e7850b-340a-4927-9066-f93a01337fed-tls-certs\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.885749 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.885724    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwg9r\" (UniqueName: \"kubernetes.io/projected/a6e7850b-340a-4927-9066-f93a01337fed-kube-api-access-zwg9r\") pod \"sub-test-model-0337e8-kserve-db67b6db7-wzkkm\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") " pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:11.935480 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:11.935441    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:12.070045 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:12.070014    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"]
Jul 28 01:53:12.073351 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:53:12.073316    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6e7850b_340a_4927_9066_f93a01337fed.slice/crio-6f4570c323ceb081799cf43e344647ef2089f6e5cc793fabc23ef5904f6701ce WatchSource:0}: Error finding container 6f4570c323ceb081799cf43e344647ef2089f6e5cc793fabc23ef5904f6701ce: Status 404 returned error can't find the container with id 6f4570c323ceb081799cf43e344647ef2089f6e5cc793fabc23ef5904f6701ce
Jul 28 01:53:13.057837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:13.057800    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" event={"ID":"a6e7850b-340a-4927-9066-f93a01337fed","Type":"ContainerStarted","Data":"ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c"}
Jul 28 01:53:13.058261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:13.057846    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" event={"ID":"a6e7850b-340a-4927-9066-f93a01337fed","Type":"ContainerStarted","Data":"6f4570c323ceb081799cf43e344647ef2089f6e5cc793fabc23ef5904f6701ce"}
Jul 28 01:53:13.078168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:13.078114    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" podStartSLOduration=2.078099317 podStartE2EDuration="2.078099317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:53:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:53:13.076209897 +0000 UTC m=+2929.087448543" watchObservedRunningTime="2026-07-28 01:53:13.078099317 +0000 UTC m=+2929.089337925"
Jul 28 01:53:15.740930 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:15.740893    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:53:21.936180 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:21.936129    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:21.936180 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:21.936195    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:21.948900 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:21.948869    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:22.111915 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:22.111873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:53:24.923813 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:24.923781    2566 scope.go:117] "RemoveContainer" containerID="5d5fbf98934f4729de28c4702711a61e108e5d21f9a762270b0de5720ca79a9e"
Jul 28 01:53:32.229066 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.229031    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"]
Jul 28 01:53:32.232467 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.232449    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.234901 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.234876    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"kube-root-ca.crt\""
Jul 28 01:53:32.235213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.235195    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"openshift-service-ca.crt\""
Jul 28 01:53:32.235999 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.235979    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"default-dockercfg-qjqfd\""
Jul 28 01:53:32.236104 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.235997    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-44b320\"/\"sub-test-model-44b320-kserve-self-signed-certs\""
Jul 28 01:53:32.237024 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.237007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-model-cache\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.237107 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.237045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghwz9\" (UniqueName: \"kubernetes.io/projected/7dfb4353-b86a-42d8-8d40-614745d2a9a4-kube-api-access-ghwz9\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.237146 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.237099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-home\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.237216 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.237198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-dshm\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.237258 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.237227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tls-certs\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.237342 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.237322    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tmp-dir\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.241470 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.241452    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"]
Jul 28 01:53:32.337729 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.337691    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-home\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.337936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.337751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-dshm\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.337936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.337774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tls-certs\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.337936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.337831    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tmp-dir\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.338139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.337957    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-model-cache\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.338139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.338020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghwz9\" (UniqueName: \"kubernetes.io/projected/7dfb4353-b86a-42d8-8d40-614745d2a9a4-kube-api-access-ghwz9\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.338288 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.338262    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tmp-dir\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.338288 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.338277    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-home\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.338443 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.338341    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-model-cache\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.340277 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.340253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-dshm\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.340482 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.340468    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tls-certs\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.346956 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.346934    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghwz9\" (UniqueName: \"kubernetes.io/projected/7dfb4353-b86a-42d8-8d40-614745d2a9a4-kube-api-access-ghwz9\") pod \"sub-test-model-44b320-kserve-65d4c888c7-w6dht\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") " pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.542723 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.542632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:32.683171 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:32.683143    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"]
Jul 28 01:53:32.684930 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:53:32.684888    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dfb4353_b86a_42d8_8d40_614745d2a9a4.slice/crio-1338ce83a5bf1f1ef3a9aa6264d6ea9f0e76cff811f4987803846bd0b5af9d82 WatchSource:0}: Error finding container 1338ce83a5bf1f1ef3a9aa6264d6ea9f0e76cff811f4987803846bd0b5af9d82: Status 404 returned error can't find the container with id 1338ce83a5bf1f1ef3a9aa6264d6ea9f0e76cff811f4987803846bd0b5af9d82
Jul 28 01:53:33.157625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:33.157593    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht" event={"ID":"7dfb4353-b86a-42d8-8d40-614745d2a9a4","Type":"ContainerStarted","Data":"c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9"}
Jul 28 01:53:33.157625 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:33.157628    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht" event={"ID":"7dfb4353-b86a-42d8-8d40-614745d2a9a4","Type":"ContainerStarted","Data":"1338ce83a5bf1f1ef3a9aa6264d6ea9f0e76cff811f4987803846bd0b5af9d82"}
Jul 28 01:53:33.181009 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:33.178881    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht" podStartSLOduration=1.17886124 podStartE2EDuration="1.17886124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:53:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:53:33.175685789 +0000 UTC m=+2949.186924412" watchObservedRunningTime="2026-07-28 01:53:33.17886124 +0000 UTC m=+2949.190099850"
Jul 28 01:53:35.640426 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:35.640392    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:53:42.543562 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:42.543516    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:42.544094 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:42.543581    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:42.560809 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:42.560782    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:53:43.221299 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:53:43.221269    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:54:11.641765 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:11.641727    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:54:30.834900 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:30.834863    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:54:36.862858 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:36.862826    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"]
Jul 28 01:54:37.651192 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:37.651156    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"]
Jul 28 01:54:47.930659 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:47.930620    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:54:51.882799 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:51.882752    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" podUID="a6e7850b-340a-4927-9066-f93a01337fed" containerName="main" containerID="cri-o://ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c" gracePeriod=45
Jul 28 01:54:52.137186 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.137120    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:54:52.223919 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.223864    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-tmp-dir\") pod \"a6e7850b-340a-4927-9066-f93a01337fed\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") "
Jul 28 01:54:52.223919 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.223936    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6e7850b-340a-4927-9066-f93a01337fed-tls-certs\") pod \"a6e7850b-340a-4927-9066-f93a01337fed\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") "
Jul 28 01:54:52.224213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.223988    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-model-cache\") pod \"a6e7850b-340a-4927-9066-f93a01337fed\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") "
Jul 28 01:54:52.224213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.224008    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwg9r\" (UniqueName: \"kubernetes.io/projected/a6e7850b-340a-4927-9066-f93a01337fed-kube-api-access-zwg9r\") pod \"a6e7850b-340a-4927-9066-f93a01337fed\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") "
Jul 28 01:54:52.224213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.224045    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-dshm\") pod \"a6e7850b-340a-4927-9066-f93a01337fed\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") "
Jul 28 01:54:52.224213 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.224085    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-home\") pod \"a6e7850b-340a-4927-9066-f93a01337fed\" (UID: \"a6e7850b-340a-4927-9066-f93a01337fed\") "
Jul 28 01:54:52.224440 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.224230    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6e7850b-340a-4927-9066-f93a01337fed" (UID: "a6e7850b-340a-4927-9066-f93a01337fed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:52.224440 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.224240    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6e7850b-340a-4927-9066-f93a01337fed" (UID: "a6e7850b-340a-4927-9066-f93a01337fed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:52.224440 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.224413    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-home" (OuterVolumeSpecName: "home") pod "a6e7850b-340a-4927-9066-f93a01337fed" (UID: "a6e7850b-340a-4927-9066-f93a01337fed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:52.226339 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.226313    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e7850b-340a-4927-9066-f93a01337fed-kube-api-access-zwg9r" (OuterVolumeSpecName: "kube-api-access-zwg9r") pod "a6e7850b-340a-4927-9066-f93a01337fed" (UID: "a6e7850b-340a-4927-9066-f93a01337fed"). InnerVolumeSpecName "kube-api-access-zwg9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:54:52.226339 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.226327    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-dshm" (OuterVolumeSpecName: "dshm") pod "a6e7850b-340a-4927-9066-f93a01337fed" (UID: "a6e7850b-340a-4927-9066-f93a01337fed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:52.226461 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.226363    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e7850b-340a-4927-9066-f93a01337fed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6e7850b-340a-4927-9066-f93a01337fed" (UID: "a6e7850b-340a-4927-9066-f93a01337fed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:54:52.325179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.325137    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:52.325179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.325169    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6e7850b-340a-4927-9066-f93a01337fed-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:52.325179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.325178    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:52.325179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.325187    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwg9r\" (UniqueName: \"kubernetes.io/projected/a6e7850b-340a-4927-9066-f93a01337fed-kube-api-access-zwg9r\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:52.325458 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.325196    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:52.325458 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.325205    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6e7850b-340a-4927-9066-f93a01337fed-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:52.550089 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.549997    2566 generic.go:356] "Generic (PLEG): container finished" podID="a6e7850b-340a-4927-9066-f93a01337fed" containerID="ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c" exitCode=0
Jul 28 01:54:52.550089 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.550034    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"
Jul 28 01:54:52.550089 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.550075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" event={"ID":"a6e7850b-340a-4927-9066-f93a01337fed","Type":"ContainerDied","Data":"ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c"}
Jul 28 01:54:52.550354 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.550119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" event={"ID":"a6e7850b-340a-4927-9066-f93a01337fed","Type":"ContainerDied","Data":"6f4570c323ceb081799cf43e344647ef2089f6e5cc793fabc23ef5904f6701ce"}
Jul 28 01:54:52.550354 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.550136    2566 scope.go:117] "RemoveContainer" containerID="ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c"
Jul 28 01:54:52.559811 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.559794    2566 scope.go:117] "RemoveContainer" containerID="ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c"
Jul 28 01:54:52.560107 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:54:52.560089    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c\": container with ID starting with ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c not found: ID does not exist" containerID="ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c"
Jul 28 01:54:52.560179 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.560117    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c"} err="failed to get container status \"ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c\": rpc error: code = NotFound desc = could not find container \"ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c\": container with ID starting with ee8fc741914f55a979458511afa68e9a59c583d0f0ac6517f91cbdb62a7cee3c not found: ID does not exist"
Jul 28 01:54:52.571299 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.571277    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"]
Jul 28 01:54:52.574835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.574816    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm"]
Jul 28 01:54:52.671376 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.671319    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht" podUID="7dfb4353-b86a-42d8-8d40-614745d2a9a4" containerName="main" containerID="cri-o://c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9" gracePeriod=45
Jul 28 01:54:52.920602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:52.920575    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:54:53.031526 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031496    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tmp-dir\") pod \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") "
Jul 28 01:54:53.031707 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031553    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tls-certs\") pod \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") "
Jul 28 01:54:53.031707 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031591    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-dshm\") pod \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") "
Jul 28 01:54:53.031707 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031612    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-home\") pod \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") "
Jul 28 01:54:53.031707 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031682    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-model-cache\") pod \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") "
Jul 28 01:54:53.031873 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031713    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghwz9\" (UniqueName: \"kubernetes.io/projected/7dfb4353-b86a-42d8-8d40-614745d2a9a4-kube-api-access-ghwz9\") pod \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\" (UID: \"7dfb4353-b86a-42d8-8d40-614745d2a9a4\") "
Jul 28 01:54:53.031873 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031760    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7dfb4353-b86a-42d8-8d40-614745d2a9a4" (UID: "7dfb4353-b86a-42d8-8d40-614745d2a9a4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:53.031984 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031897    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-home" (OuterVolumeSpecName: "home") pod "7dfb4353-b86a-42d8-8d40-614745d2a9a4" (UID: "7dfb4353-b86a-42d8-8d40-614745d2a9a4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:53.031984 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.031935    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-model-cache" (OuterVolumeSpecName: "model-cache") pod "7dfb4353-b86a-42d8-8d40-614745d2a9a4" (UID: "7dfb4353-b86a-42d8-8d40-614745d2a9a4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:53.032100 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.032084    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:53.032148 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.032106    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:53.032148 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.032119    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:53.034053 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.034025    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-dshm" (OuterVolumeSpecName: "dshm") pod "7dfb4353-b86a-42d8-8d40-614745d2a9a4" (UID: "7dfb4353-b86a-42d8-8d40-614745d2a9a4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:54:53.034164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.034096    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7dfb4353-b86a-42d8-8d40-614745d2a9a4" (UID: "7dfb4353-b86a-42d8-8d40-614745d2a9a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:54:53.034164 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.034119    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dfb4353-b86a-42d8-8d40-614745d2a9a4-kube-api-access-ghwz9" (OuterVolumeSpecName: "kube-api-access-ghwz9") pod "7dfb4353-b86a-42d8-8d40-614745d2a9a4" (UID: "7dfb4353-b86a-42d8-8d40-614745d2a9a4"). InnerVolumeSpecName "kube-api-access-ghwz9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:54:53.133306 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.133215    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7dfb4353-b86a-42d8-8d40-614745d2a9a4-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:53.133306 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.133246    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7dfb4353-b86a-42d8-8d40-614745d2a9a4-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:53.133306 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.133257    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghwz9\" (UniqueName: \"kubernetes.io/projected/7dfb4353-b86a-42d8-8d40-614745d2a9a4-kube-api-access-ghwz9\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:54:53.557308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.557205    2566 generic.go:356] "Generic (PLEG): container finished" podID="7dfb4353-b86a-42d8-8d40-614745d2a9a4" containerID="c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9" exitCode=0
Jul 28 01:54:53.557308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.557252    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht" event={"ID":"7dfb4353-b86a-42d8-8d40-614745d2a9a4","Type":"ContainerDied","Data":"c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9"}
Jul 28 01:54:53.557308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.557274    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht" event={"ID":"7dfb4353-b86a-42d8-8d40-614745d2a9a4","Type":"ContainerDied","Data":"1338ce83a5bf1f1ef3a9aa6264d6ea9f0e76cff811f4987803846bd0b5af9d82"}
Jul 28 01:54:53.557308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.557276    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"
Jul 28 01:54:53.557308 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.557298    2566 scope.go:117] "RemoveContainer" containerID="c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9"
Jul 28 01:54:53.568511 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.568480    2566 scope.go:117] "RemoveContainer" containerID="c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9"
Jul 28 01:54:53.569059 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:54:53.569034    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9\": container with ID starting with c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9 not found: ID does not exist" containerID="c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9"
Jul 28 01:54:53.569184 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.569067    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9"} err="failed to get container status \"c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9\": rpc error: code = NotFound desc = could not find container \"c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9\": container with ID starting with c5d67cbdb97aac5a9e52ca6bcf2ca029220fea4fc3a07b2e3f8f976483560ee9 not found: ID does not exist"
Jul 28 01:54:53.582347 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.582315    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"]
Jul 28 01:54:53.587549 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:53.587527    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-44b320/sub-test-model-44b320-kserve-65d4c888c7-w6dht"]
Jul 28 01:54:54.561356 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:54.561325    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dfb4353-b86a-42d8-8d40-614745d2a9a4" path="/var/lib/kubelet/pods/7dfb4353-b86a-42d8-8d40-614745d2a9a4/volumes"
Jul 28 01:54:54.561849 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:54.561687    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e7850b-340a-4927-9066-f93a01337fed" path="/var/lib/kubelet/pods/a6e7850b-340a-4927-9066-f93a01337fed/volumes"
Jul 28 01:54:57.102283 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:54:57.102229    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-0337e8/sub-test-model-0337e8-kserve-db67b6db7-wzkkm" podUID="a6e7850b-340a-4927-9066-f93a01337fed" containerName="main" probeResult="failure" output="Get \"https://10.133.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 01:55:00.938054 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938020    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"]
Jul 28 01:55:00.938472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938407    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dfb4353-b86a-42d8-8d40-614745d2a9a4" containerName="main"
Jul 28 01:55:00.938472 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938420    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dfb4353-b86a-42d8-8d40-614745d2a9a4" containerName="main"
Jul 28 01:55:00.938573 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938475    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e7850b-340a-4927-9066-f93a01337fed" containerName="main"
Jul 28 01:55:00.938573 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938484    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e7850b-340a-4927-9066-f93a01337fed" containerName="main"
Jul 28 01:55:00.938646 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938620    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e7850b-340a-4927-9066-f93a01337fed" containerName="main"
Jul 28 01:55:00.938646 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.938634    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dfb4353-b86a-42d8-8d40-614745d2a9a4" containerName="main"
Jul 28 01:55:00.943060 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.943041    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:00.946175 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.946155    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"kube-root-ca.crt\""
Jul 28 01:55:00.947056 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.947032    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"default-dockercfg-z24k8\""
Jul 28 01:55:00.947172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.947121    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"openshift-service-ca.crt\""
Jul 28 01:55:00.947172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.947130    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-0337e8\"/\"rate-test-model-0337e8-kserve-self-signed-certs\""
Jul 28 01:55:00.955875 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:00.955852    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"]
Jul 28 01:55:01.003936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.003899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tmp-dir\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.003936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.003938    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tls-certs\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.004172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.003960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-home\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.004172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.004019    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-dshm\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.004172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.004080    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-model-cache\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.004172 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.004101    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncs5s\" (UniqueName: \"kubernetes.io/projected/0f07368a-ec7c-40a6-87b4-61eea77ce60a-kube-api-access-ncs5s\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.104941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.104900    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-model-cache\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.104941 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.104942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncs5s\" (UniqueName: \"kubernetes.io/projected/0f07368a-ec7c-40a6-87b4-61eea77ce60a-kube-api-access-ncs5s\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105006    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tmp-dir\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tls-certs\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105044    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-home\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-dshm\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105445 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105394    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-model-cache\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105493 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tmp-dir\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.105493 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.105481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-home\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.107515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.107492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-dshm\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.107799 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.107777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tls-certs\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.113287 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.113270    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncs5s\" (UniqueName: \"kubernetes.io/projected/0f07368a-ec7c-40a6-87b4-61eea77ce60a-kube-api-access-ncs5s\") pod \"rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") " pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.257097 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.256994    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:01.394223 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.394191    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"]
Jul 28 01:55:01.396704 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:55:01.396678    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f07368a_ec7c_40a6_87b4_61eea77ce60a.slice/crio-0a51d299cf39b33a987d01d1dd5845343e8e8ac806e6fe09d72aff6c02cfa731 WatchSource:0}: Error finding container 0a51d299cf39b33a987d01d1dd5845343e8e8ac806e6fe09d72aff6c02cfa731: Status 404 returned error can't find the container with id 0a51d299cf39b33a987d01d1dd5845343e8e8ac806e6fe09d72aff6c02cfa731
Jul 28 01:55:01.595422 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.595383    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll" event={"ID":"0f07368a-ec7c-40a6-87b4-61eea77ce60a","Type":"ContainerStarted","Data":"c90b7a6a50ca145ed5954b70076a305bb433116b89a2e8bf1e29546c3d49c36d"}
Jul 28 01:55:01.595422 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.595427    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll" event={"ID":"0f07368a-ec7c-40a6-87b4-61eea77ce60a","Type":"ContainerStarted","Data":"0a51d299cf39b33a987d01d1dd5845343e8e8ac806e6fe09d72aff6c02cfa731"}
Jul 28 01:55:01.613105 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:01.613053    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll" podStartSLOduration=1.613037618 podStartE2EDuration="1.613037618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:55:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:55:01.61179871 +0000 UTC m=+3037.623037319" watchObservedRunningTime="2026-07-28 01:55:01.613037618 +0000 UTC m=+3037.624276227"
Jul 28 01:55:04.040637 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:04.040602    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:55:11.257379 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:11.257338    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:11.257876 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:11.257396    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:11.270135 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:11.270106    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:11.654373 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:11.654341    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:55:34.841647 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:55:34.841611    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:56:06.531832 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:06.531791    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:56:20.432302 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:20.432263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:56:34.752667 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:34.752625    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:56:38.959445 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:38.959408    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"]
Jul 28 01:56:49.938041 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:49.937934    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:56:53.979728 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:53.979683    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll" podUID="0f07368a-ec7c-40a6-87b4-61eea77ce60a" containerName="main" containerID="cri-o://c90b7a6a50ca145ed5954b70076a305bb433116b89a2e8bf1e29546c3d49c36d" gracePeriod=45
Jul 28 01:56:54.131871 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.131832    2566 generic.go:356] "Generic (PLEG): container finished" podID="0f07368a-ec7c-40a6-87b4-61eea77ce60a" containerID="c90b7a6a50ca145ed5954b70076a305bb433116b89a2e8bf1e29546c3d49c36d" exitCode=0
Jul 28 01:56:54.132064 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.131897    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll" event={"ID":"0f07368a-ec7c-40a6-87b4-61eea77ce60a","Type":"ContainerDied","Data":"c90b7a6a50ca145ed5954b70076a305bb433116b89a2e8bf1e29546c3d49c36d"}
Jul 28 01:56:54.236844 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.236786    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:56:54.326948 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.326912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tmp-dir\") pod \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") "
Jul 28 01:56:54.327154 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.326998    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-home\") pod \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") "
Jul 28 01:56:54.327154 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327024    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncs5s\" (UniqueName: \"kubernetes.io/projected/0f07368a-ec7c-40a6-87b4-61eea77ce60a-kube-api-access-ncs5s\") pod \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") "
Jul 28 01:56:54.327154 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327081    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-dshm\") pod \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") "
Jul 28 01:56:54.327337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327174    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tls-certs\") pod \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") "
Jul 28 01:56:54.327337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327217    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0f07368a-ec7c-40a6-87b4-61eea77ce60a" (UID: "0f07368a-ec7c-40a6-87b4-61eea77ce60a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:54.327337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327229    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-model-cache\") pod \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\" (UID: \"0f07368a-ec7c-40a6-87b4-61eea77ce60a\") "
Jul 28 01:56:54.327337 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327321    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-home" (OuterVolumeSpecName: "home") pod "0f07368a-ec7c-40a6-87b4-61eea77ce60a" (UID: "0f07368a-ec7c-40a6-87b4-61eea77ce60a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:54.327506 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327436    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-model-cache" (OuterVolumeSpecName: "model-cache") pod "0f07368a-ec7c-40a6-87b4-61eea77ce60a" (UID: "0f07368a-ec7c-40a6-87b4-61eea77ce60a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:54.327563 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327546    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-model-cache\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.327601 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327575    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tmp-dir\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.327601 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.327592    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-home\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.329495 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.329459    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-dshm" (OuterVolumeSpecName: "dshm") pod "0f07368a-ec7c-40a6-87b4-61eea77ce60a" (UID: "0f07368a-ec7c-40a6-87b4-61eea77ce60a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 01:56:54.329495 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.329473    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f07368a-ec7c-40a6-87b4-61eea77ce60a-kube-api-access-ncs5s" (OuterVolumeSpecName: "kube-api-access-ncs5s") pod "0f07368a-ec7c-40a6-87b4-61eea77ce60a" (UID: "0f07368a-ec7c-40a6-87b4-61eea77ce60a"). InnerVolumeSpecName "kube-api-access-ncs5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:56:54.329495 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.329459    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0f07368a-ec7c-40a6-87b4-61eea77ce60a" (UID: "0f07368a-ec7c-40a6-87b4-61eea77ce60a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 01:56:54.428507 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.428461    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f07368a-ec7c-40a6-87b4-61eea77ce60a-dshm\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.428507 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.428506    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f07368a-ec7c-40a6-87b4-61eea77ce60a-tls-certs\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:56:54.428722 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:54.428524    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncs5s\" (UniqueName: \"kubernetes.io/projected/0f07368a-ec7c-40a6-87b4-61eea77ce60a-kube-api-access-ncs5s\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:56:55.139009 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:55.138954    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"
Jul 28 01:56:55.139464 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:55.139005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll" event={"ID":"0f07368a-ec7c-40a6-87b4-61eea77ce60a","Type":"ContainerDied","Data":"0a51d299cf39b33a987d01d1dd5845343e8e8ac806e6fe09d72aff6c02cfa731"}
Jul 28 01:56:55.139464 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:55.139059    2566 scope.go:117] "RemoveContainer" containerID="c90b7a6a50ca145ed5954b70076a305bb433116b89a2e8bf1e29546c3d49c36d"
Jul 28 01:56:55.163733 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:55.163707    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"]
Jul 28 01:56:55.167956 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:55.167934    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-0337e8/rate-test-model-0337e8-kserve-bbd7ddd59-jq5ll"]
Jul 28 01:56:56.560586 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:56:56.560553    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f07368a-ec7c-40a6-87b4-61eea77ce60a" path="/var/lib/kubelet/pods/0f07368a-ec7c-40a6-87b4-61eea77ce60a/volumes"
Jul 28 01:57:03.051171 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.051127    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"]
Jul 28 01:57:03.051654 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.051632    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f07368a-ec7c-40a6-87b4-61eea77ce60a" containerName="main"
Jul 28 01:57:03.051654 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.051651    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f07368a-ec7c-40a6-87b4-61eea77ce60a" containerName="main"
Jul 28 01:57:03.051773 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.051764    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f07368a-ec7c-40a6-87b4-61eea77ce60a" containerName="main"
Jul 28 01:57:03.054768 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.054743    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.057921 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.057901    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8a33ed\""
Jul 28 01:57:03.058107 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.058089    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8a33ed-dockercfg-f5d9k\""
Jul 28 01:57:03.068445 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.068419    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"]
Jul 28 01:57:03.080483 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.080453    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"]
Jul 28 01:57:03.083515 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.083499    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.094559 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.094540    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"]
Jul 28 01:57:03.103036 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.103014    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crpvb\" (UniqueName: \"kubernetes.io/projected/533ab225-b26b-4d8c-91b7-d53fb13f90a1-kube-api-access-crpvb\") pod \"payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.103132 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.103068    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533ab225-b26b-4d8c-91b7-d53fb13f90a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.204254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.204213    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.204476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.204289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crpvb\" (UniqueName: \"kubernetes.io/projected/533ab225-b26b-4d8c-91b7-d53fb13f90a1-kube-api-access-crpvb\") pod \"payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.204476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.204351    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6wj5\" (UniqueName: \"kubernetes.io/projected/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-kube-api-access-z6wj5\") pod \"payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.204476 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.204379    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533ab225-b26b-4d8c-91b7-d53fb13f90a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.205157 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.205133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533ab225-b26b-4d8c-91b7-d53fb13f90a1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.213844 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.213816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crpvb\" (UniqueName: \"kubernetes.io/projected/533ab225-b26b-4d8c-91b7-d53fb13f90a1-kube-api-access-crpvb\") pod \"payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.305501 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.305406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6wj5\" (UniqueName: \"kubernetes.io/projected/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-kube-api-access-z6wj5\") pod \"payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.305501 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.305473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.306284 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.306265    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.313585 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.313561    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6wj5\" (UniqueName: \"kubernetes.io/projected/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-kube-api-access-z6wj5\") pod \"payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.365309 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.365272    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:03.395070 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.395035    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:03.522321 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.522289    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"]
Jul 28 01:57:03.524052 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:57:03.524004    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533ab225_b26b_4d8c_91b7_d53fb13f90a1.slice/crio-3f0f998f971c1fc7ae66c2151af1448a565fdd5fc105296b793caf377a9a6eff WatchSource:0}: Error finding container 3f0f998f971c1fc7ae66c2151af1448a565fdd5fc105296b793caf377a9a6eff: Status 404 returned error can't find the container with id 3f0f998f971c1fc7ae66c2151af1448a565fdd5fc105296b793caf377a9a6eff
Jul 28 01:57:03.758352 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:03.758323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"]
Jul 28 01:57:03.760169 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:57:03.760144    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3a69f7_a977_4eed_a2c3_6c57918b3c66.slice/crio-c2b8c15777d164939e15d6b8c30ce6c16e93c2765ee19321002b58fcf856d468 WatchSource:0}: Error finding container c2b8c15777d164939e15d6b8c30ce6c16e93c2765ee19321002b58fcf856d468: Status 404 returned error can't find the container with id c2b8c15777d164939e15d6b8c30ce6c16e93c2765ee19321002b58fcf856d468
Jul 28 01:57:04.182849 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.182807    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq" event={"ID":"ed3a69f7-a977-4eed-a2c3-6c57918b3c66","Type":"ContainerStarted","Data":"e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e"}
Jul 28 01:57:04.183398 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.182881    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq" event={"ID":"ed3a69f7-a977-4eed-a2c3-6c57918b3c66","Type":"ContainerStarted","Data":"c2b8c15777d164939e15d6b8c30ce6c16e93c2765ee19321002b58fcf856d468"}
Jul 28 01:57:04.183398 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.182909    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:04.184399 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.184370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr" event={"ID":"533ab225-b26b-4d8c-91b7-d53fb13f90a1","Type":"ContainerStarted","Data":"eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0"}
Jul 28 01:57:04.184523 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.184424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr" event={"ID":"533ab225-b26b-4d8c-91b7-d53fb13f90a1","Type":"ContainerStarted","Data":"3f0f998f971c1fc7ae66c2151af1448a565fdd5fc105296b793caf377a9a6eff"}
Jul 28 01:57:04.184523 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.184464    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:04.201196 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.201149    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq" podStartSLOduration=1.201134948 podStartE2EDuration="1.201134948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:04.199111581 +0000 UTC m=+3160.210350209" watchObservedRunningTime="2026-07-28 01:57:04.201134948 +0000 UTC m=+3160.212373556"
Jul 28 01:57:04.216943 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:04.216882    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr" podStartSLOduration=1.216858212 podStartE2EDuration="1.216858212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:04.215224447 +0000 UTC m=+3160.226463077" watchObservedRunningTime="2026-07-28 01:57:04.216858212 +0000 UTC m=+3160.228096822"
Jul 28 01:57:10.736008 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:10.735956    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:57:15.191342 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:15.191310    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:57:15.191798 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:15.191379    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:57:32.574795 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.574757    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"]
Jul 28 01:57:32.586244 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.586211    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.587218 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.587149    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"]
Jul 28 01:57:32.589209 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.589186    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c430e4-dockercfg-ptqn2\""
Jul 28 01:57:32.589329 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.589186    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c430e4\""
Jul 28 01:57:32.590664 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.590644    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"]
Jul 28 01:57:32.590776 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.590726    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.599003 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.598961    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"]
Jul 28 01:57:32.762503 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.762461    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516e29f-3f85-4436-957d-fcfd64c42582-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.762704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.762512    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qz52\" (UniqueName: \"kubernetes.io/projected/775c435b-ddde-4d68-9c4c-8019c542191c-kube-api-access-4qz52\") pod \"payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.762704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.762529    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtv9f\" (UniqueName: \"kubernetes.io/projected/c516e29f-3f85-4436-957d-fcfd64c42582-kube-api-access-xtv9f\") pod \"payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.762704 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.762639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775c435b-ddde-4d68-9c4c-8019c542191c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.863771 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.863738    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775c435b-ddde-4d68-9c4c-8019c542191c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.863940 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.863793    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516e29f-3f85-4436-957d-fcfd64c42582-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.863940 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.863833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qz52\" (UniqueName: \"kubernetes.io/projected/775c435b-ddde-4d68-9c4c-8019c542191c-kube-api-access-4qz52\") pod \"payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.863940 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.863856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtv9f\" (UniqueName: \"kubernetes.io/projected/c516e29f-3f85-4436-957d-fcfd64c42582-kube-api-access-xtv9f\") pod \"payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.864584 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.864564    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775c435b-ddde-4d68-9c4c-8019c542191c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.864626 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.864567    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516e29f-3f85-4436-957d-fcfd64c42582-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.873216 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.873175    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtv9f\" (UniqueName: \"kubernetes.io/projected/c516e29f-3f85-4436-957d-fcfd64c42582-kube-api-access-xtv9f\") pod \"payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:32.873216 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.873212    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qz52\" (UniqueName: \"kubernetes.io/projected/775c435b-ddde-4d68-9c4c-8019c542191c-kube-api-access-4qz52\") pod \"payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.900950 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.900918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:32.906908 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:32.906889    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:33.053468 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.053439    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"]
Jul 28 01:57:33.055545 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:57:33.055523    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775c435b_ddde_4d68_9c4c_8019c542191c.slice/crio-6085bb9cfb12800c866c8cfc83b9678cb399a886d96317a262602141a64ef96e WatchSource:0}: Error finding container 6085bb9cfb12800c866c8cfc83b9678cb399a886d96317a262602141a64ef96e: Status 404 returned error can't find the container with id 6085bb9cfb12800c866c8cfc83b9678cb399a886d96317a262602141a64ef96e
Jul 28 01:57:33.279478 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.279450    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"]
Jul 28 01:57:33.281029 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:57:33.281004    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc516e29f_3f85_4436_957d_fcfd64c42582.slice/crio-e3a0d8053fb61bcec0cb3d1627e77221eed18c9edc0aef33613ae782d66a1713 WatchSource:0}: Error finding container e3a0d8053fb61bcec0cb3d1627e77221eed18c9edc0aef33613ae782d66a1713: Status 404 returned error can't find the container with id e3a0d8053fb61bcec0cb3d1627e77221eed18c9edc0aef33613ae782d66a1713
Jul 28 01:57:33.326098 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.326069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6" event={"ID":"c516e29f-3f85-4436-957d-fcfd64c42582","Type":"ContainerStarted","Data":"e3a0d8053fb61bcec0cb3d1627e77221eed18c9edc0aef33613ae782d66a1713"}
Jul 28 01:57:33.327535 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.327513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd" event={"ID":"775c435b-ddde-4d68-9c4c-8019c542191c","Type":"ContainerStarted","Data":"609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720"}
Jul 28 01:57:33.327658 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.327552    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd" event={"ID":"775c435b-ddde-4d68-9c4c-8019c542191c","Type":"ContainerStarted","Data":"6085bb9cfb12800c866c8cfc83b9678cb399a886d96317a262602141a64ef96e"}
Jul 28 01:57:33.327658 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.327650    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:33.344001 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:33.343938    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd" podStartSLOduration=1.343923001 podStartE2EDuration="1.343923001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:33.343419713 +0000 UTC m=+3189.354658333" watchObservedRunningTime="2026-07-28 01:57:33.343923001 +0000 UTC m=+3189.355161610"
Jul 28 01:57:34.334780 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:34.334741    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6" event={"ID":"c516e29f-3f85-4436-957d-fcfd64c42582","Type":"ContainerStarted","Data":"16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927"}
Jul 28 01:57:34.335246 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:34.334939    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:34.352257 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:34.352175    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6" podStartSLOduration=2.35215622 podStartE2EDuration="2.35215622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:57:34.351313075 +0000 UTC m=+3190.362551710" watchObservedRunningTime="2026-07-28 01:57:34.35215622 +0000 UTC m=+3190.363394830"
Jul 28 01:57:41.465928 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:41.465890    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:57:44.337621 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:44.337589    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 01:57:45.341862 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:45.341823    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 01:57:53.185137 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:57:53.185107    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6_c516e29f-3f85-4436-957d-fcfd64c42582/payload-processing/0.log"
Jul 28 01:58:06.341035 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:06.340996    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:58:33.636640 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:33.636601    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:58:42.979566 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:42.979536    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq_ed3a69f7-a977-4eed-a2c3-6c57918b3c66/payload-processing/0.log"
Jul 28 01:58:43.207059 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:43.207020    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6_c516e29f-3f85-4436-957d-fcfd64c42582/payload-processing/0.log"
Jul 28 01:58:49.627368 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.627331    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"]
Jul 28 01:58:49.630922 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.630899    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.633425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.633402    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9e1b27-dockercfg-kd7mn\""
Jul 28 01:58:49.633709 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.633690    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9e1b27\""
Jul 28 01:58:49.640933 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.640905    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"]
Jul 28 01:58:49.678892 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.678859    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"]
Jul 28 01:58:49.682392 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.682377    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.695183 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.695156    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"]
Jul 28 01:58:49.747439 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.747400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6b6289-fd25-4985-82f6-d02d880611e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.747613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.747447    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zcm7\" (UniqueName: \"kubernetes.io/projected/bc6b6289-fd25-4985-82f6-d02d880611e3-kube-api-access-6zcm7\") pod \"payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.747613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.747466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.747613 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.747538    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8rvj\" (UniqueName: \"kubernetes.io/projected/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-kube-api-access-p8rvj\") pod \"payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.848057 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.848023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6b6289-fd25-4985-82f6-d02d880611e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.848229 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.848066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zcm7\" (UniqueName: \"kubernetes.io/projected/bc6b6289-fd25-4985-82f6-d02d880611e3-kube-api-access-6zcm7\") pod \"payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.848229 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.848085    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.848229 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.848217    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8rvj\" (UniqueName: \"kubernetes.io/projected/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-kube-api-access-p8rvj\") pod \"payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.848791 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.848773    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6b6289-fd25-4985-82f6-d02d880611e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.848834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.848806    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.856261 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.856237    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8rvj\" (UniqueName: \"kubernetes.io/projected/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-kube-api-access-p8rvj\") pod \"payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:49.856396 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.856381    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zcm7\" (UniqueName: \"kubernetes.io/projected/bc6b6289-fd25-4985-82f6-d02d880611e3-kube-api-access-6zcm7\") pod \"payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.942908 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.942817    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:49.992393 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:49.992171    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:50.102869 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.102834    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"]
Jul 28 01:58:50.110327 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:58:50.110294    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6b6289_fd25_4985_82f6_d02d880611e3.slice/crio-401d2e1f3de9515fc1523d688fed32e593cf79b6a16a41cb95c0510efb80798b WatchSource:0}: Error finding container 401d2e1f3de9515fc1523d688fed32e593cf79b6a16a41cb95c0510efb80798b: Status 404 returned error can't find the container with id 401d2e1f3de9515fc1523d688fed32e593cf79b6a16a41cb95c0510efb80798b
Jul 28 01:58:50.146741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.146715    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"]
Jul 28 01:58:50.148559 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:58:50.148534    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a0f4cb_38c8_456e_bcc7_ec85be95de7b.slice/crio-5862d098a9b2d42dd0758d7a7345d01e9f19e9c9faa89125ad3b7d909b41d679 WatchSource:0}: Error finding container 5862d098a9b2d42dd0758d7a7345d01e9f19e9c9faa89125ad3b7d909b41d679: Status 404 returned error can't find the container with id 5862d098a9b2d42dd0758d7a7345d01e9f19e9c9faa89125ad3b7d909b41d679
Jul 28 01:58:50.715951 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.715904    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh" event={"ID":"bc6b6289-fd25-4985-82f6-d02d880611e3","Type":"ContainerStarted","Data":"86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230"}
Jul 28 01:58:50.716477 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.716008    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:58:50.716477 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.716038    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh" event={"ID":"bc6b6289-fd25-4985-82f6-d02d880611e3","Type":"ContainerStarted","Data":"401d2e1f3de9515fc1523d688fed32e593cf79b6a16a41cb95c0510efb80798b"}
Jul 28 01:58:50.717683 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.717651    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr" event={"ID":"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b","Type":"ContainerStarted","Data":"0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58"}
Jul 28 01:58:50.717815 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.717711    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:58:50.717815 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.717732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr" event={"ID":"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b","Type":"ContainerStarted","Data":"5862d098a9b2d42dd0758d7a7345d01e9f19e9c9faa89125ad3b7d909b41d679"}
Jul 28 01:58:50.733767 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.733712    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh" podStartSLOduration=1.733696922 podStartE2EDuration="1.733696922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:58:50.731422734 +0000 UTC m=+3266.742661392" watchObservedRunningTime="2026-07-28 01:58:50.733696922 +0000 UTC m=+3266.744935528"
Jul 28 01:58:50.746777 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.746723    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr" podStartSLOduration=1.746703679 podStartE2EDuration="1.746703679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:58:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:58:50.746204286 +0000 UTC m=+3266.757442921" watchObservedRunningTime="2026-07-28 01:58:50.746703679 +0000 UTC m=+3266.757942293"
Jul 28 01:58:50.938495 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.938462    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk7kd"]
Jul 28 01:58:50.942084 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.942068    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:58:50.944516 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.944491    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-8nhdf\""
Jul 28 01:58:50.951411 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:50.951390    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk7kd"]
Jul 28 01:58:51.059835 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.059750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cbgg\" (UniqueName: \"kubernetes.io/projected/1f3f19df-f60f-4049-b562-4f95a2967849-kube-api-access-7cbgg\") pod \"kuadrant-operator-catalog-lk7kd\" (UID: \"1f3f19df-f60f-4049-b562-4f95a2967849\") " pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:58:51.161224 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.161186    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cbgg\" (UniqueName: \"kubernetes.io/projected/1f3f19df-f60f-4049-b562-4f95a2967849-kube-api-access-7cbgg\") pod \"kuadrant-operator-catalog-lk7kd\" (UID: \"1f3f19df-f60f-4049-b562-4f95a2967849\") " pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:58:51.169775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.169739    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cbgg\" (UniqueName: \"kubernetes.io/projected/1f3f19df-f60f-4049-b562-4f95a2967849-kube-api-access-7cbgg\") pod \"kuadrant-operator-catalog-lk7kd\" (UID: \"1f3f19df-f60f-4049-b562-4f95a2967849\") " pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:58:51.252988 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.252927    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:58:51.384759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.384731    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk7kd"]
Jul 28 01:58:51.387499 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:58:51.387473    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f3f19df_f60f_4049_b562_4f95a2967849.slice/crio-2548593426ee78ecef781c120ebdef9558a40f7e1d6fcd1766b901ebc5fb08de WatchSource:0}: Error finding container 2548593426ee78ecef781c120ebdef9558a40f7e1d6fcd1766b901ebc5fb08de: Status 404 returned error can't find the container with id 2548593426ee78ecef781c120ebdef9558a40f7e1d6fcd1766b901ebc5fb08de
Jul 28 01:58:51.389304 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.389288    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 01:58:51.727680 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:51.727646    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd" event={"ID":"1f3f19df-f60f-4049-b562-4f95a2967849","Type":"ContainerStarted","Data":"2548593426ee78ecef781c120ebdef9558a40f7e1d6fcd1766b901ebc5fb08de"}
Jul 28 01:58:52.702834 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:52.702782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:58:53.739942 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:53.739907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd" event={"ID":"1f3f19df-f60f-4049-b562-4f95a2967849","Type":"ContainerStarted","Data":"97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6"}
Jul 28 01:58:53.756629 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:58:53.756577    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd" podStartSLOduration=1.8217568910000002 podStartE2EDuration="3.756561914s" totalImagesPullingTime="1.934805023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:58:53.755535404 +0000 UTC m=+3269.766774035" watchObservedRunningTime="2026-07-28 01:58:53.756561914 +0000 UTC m=+3269.767800521"
Jul 28 01:59:01.253201 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.253166    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:59:01.253641 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.253404    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:59:01.275267 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.275244    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:59:01.730313 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.730284    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:59:01.730485 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.730360    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:59:01.800771 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.800745    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:59:01.851650 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:01.851618    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk7kd"]
Jul 28 01:59:02.397872 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.397837    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"]
Jul 28 01:59:02.402487 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.402467    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:02.407253 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.407224    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:59:02.407382 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.407359    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:59:02.410098 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.410072    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"]
Jul 28 01:59:02.569808 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.569729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3703e-57ae-4f7e-b200-4894ccbc13d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m\" (UID: \"7ae3703e-57ae-4f7e-b200-4894ccbc13d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:02.670425 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.670392    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3703e-57ae-4f7e-b200-4894ccbc13d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m\" (UID: \"7ae3703e-57ae-4f7e-b200-4894ccbc13d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:02.671023 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.671003    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3703e-57ae-4f7e-b200-4894ccbc13d8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m\" (UID: \"7ae3703e-57ae-4f7e-b200-4894ccbc13d8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:02.713880 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.713846    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:02.857241 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:02.857214    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"]
Jul 28 01:59:02.859664 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:59:02.859638    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ae3703e_57ae_4f7e_b200_4894ccbc13d8.slice/crio-297f7557bd34d2755cf7e342770cbe3454df2173c4e3df154edc5c22f6382902 WatchSource:0}: Error finding container 297f7557bd34d2755cf7e342770cbe3454df2173c4e3df154edc5c22f6382902: Status 404 returned error can't find the container with id 297f7557bd34d2755cf7e342770cbe3454df2173c4e3df154edc5c22f6382902
Jul 28 01:59:03.801355 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:03.801259    2566 generic.go:356] "Generic (PLEG): container finished" podID="7ae3703e-57ae-4f7e-b200-4894ccbc13d8" containerID="d980d9d8489614f08389a81608928f2cd894d3209640c50a1741375dd0628a0c" exitCode=0
Jul 28 01:59:03.801774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:03.801347    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m" event={"ID":"7ae3703e-57ae-4f7e-b200-4894ccbc13d8","Type":"ContainerDied","Data":"d980d9d8489614f08389a81608928f2cd894d3209640c50a1741375dd0628a0c"}
Jul 28 01:59:03.801774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:03.801401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m" event={"ID":"7ae3703e-57ae-4f7e-b200-4894ccbc13d8","Type":"ContainerStarted","Data":"297f7557bd34d2755cf7e342770cbe3454df2173c4e3df154edc5c22f6382902"}
Jul 28 01:59:03.801774 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:03.801436    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd" podUID="1f3f19df-f60f-4049-b562-4f95a2967849" containerName="registry-server" containerID="cri-o://97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6" gracePeriod=2
Jul 28 01:59:04.049112 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.049082    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:59:04.186644 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.186610    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cbgg\" (UniqueName: \"kubernetes.io/projected/1f3f19df-f60f-4049-b562-4f95a2967849-kube-api-access-7cbgg\") pod \"1f3f19df-f60f-4049-b562-4f95a2967849\" (UID: \"1f3f19df-f60f-4049-b562-4f95a2967849\") "
Jul 28 01:59:04.189063 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.189040    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f3f19df-f60f-4049-b562-4f95a2967849-kube-api-access-7cbgg" (OuterVolumeSpecName: "kube-api-access-7cbgg") pod "1f3f19df-f60f-4049-b562-4f95a2967849" (UID: "1f3f19df-f60f-4049-b562-4f95a2967849"). InnerVolumeSpecName "kube-api-access-7cbgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:04.287413 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.287379    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cbgg\" (UniqueName: \"kubernetes.io/projected/1f3f19df-f60f-4049-b562-4f95a2967849-kube-api-access-7cbgg\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:04.808040 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.808003    2566 generic.go:356] "Generic (PLEG): container finished" podID="1f3f19df-f60f-4049-b562-4f95a2967849" containerID="97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6" exitCode=0
Jul 28 01:59:04.808572 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.808073    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd"
Jul 28 01:59:04.808572 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.808087    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd" event={"ID":"1f3f19df-f60f-4049-b562-4f95a2967849","Type":"ContainerDied","Data":"97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6"}
Jul 28 01:59:04.808572 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.808163    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lk7kd" event={"ID":"1f3f19df-f60f-4049-b562-4f95a2967849","Type":"ContainerDied","Data":"2548593426ee78ecef781c120ebdef9558a40f7e1d6fcd1766b901ebc5fb08de"}
Jul 28 01:59:04.808572 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.808183    2566 scope.go:117] "RemoveContainer" containerID="97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6"
Jul 28 01:59:04.821554 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.821510    2566 scope.go:117] "RemoveContainer" containerID="97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6"
Jul 28 01:59:04.821851 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:59:04.821826    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6\": container with ID starting with 97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6 not found: ID does not exist" containerID="97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6"
Jul 28 01:59:04.821957 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.821858    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6"} err="failed to get container status \"97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6\": rpc error: code = NotFound desc = could not find container \"97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6\": container with ID starting with 97584b00991ebc356ca141f7177a2b40b3dcf05a6caebacbd8f35d7463afe1a6 not found: ID does not exist"
Jul 28 01:59:04.827168 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.827143    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk7kd"]
Jul 28 01:59:04.831781 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.831755    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lk7kd"]
Jul 28 01:59:04.954078 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:04.954052    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:05.093151 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:05.093129    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3703e-57ae-4f7e-b200-4894ccbc13d8-maas-api-service-ca\") pod \"7ae3703e-57ae-4f7e-b200-4894ccbc13d8\" (UID: \"7ae3703e-57ae-4f7e-b200-4894ccbc13d8\") "
Jul 28 01:59:05.093501 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:05.093478    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ae3703e-57ae-4f7e-b200-4894ccbc13d8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7ae3703e-57ae-4f7e-b200-4894ccbc13d8" (UID: "7ae3703e-57ae-4f7e-b200-4894ccbc13d8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:05.194533 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:05.194495    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ae3703e-57ae-4f7e-b200-4894ccbc13d8-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:05.818310 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:05.818284    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"
Jul 28 01:59:05.818776 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:05.818323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m" event={"ID":"7ae3703e-57ae-4f7e-b200-4894ccbc13d8","Type":"ContainerDied","Data":"297f7557bd34d2755cf7e342770cbe3454df2173c4e3df154edc5c22f6382902"}
Jul 28 01:59:05.818776 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:05.818363    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="297f7557bd34d2755cf7e342770cbe3454df2173c4e3df154edc5c22f6382902"
Jul 28 01:59:06.561149 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:06.561120    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f3f19df-f60f-4049-b562-4f95a2967849" path="/var/lib/kubelet/pods/1f3f19df-f60f-4049-b562-4f95a2967849/volumes"
Jul 28 01:59:11.326451 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.326408    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"]
Jul 28 01:59:11.326936 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.326675    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh" podUID="bc6b6289-fd25-4985-82f6-d02d880611e3" containerName="payload-pre-processing" containerID="cri-o://86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230" gracePeriod=30
Jul 28 01:59:11.329837 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.329809    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"]
Jul 28 01:59:11.330115 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.330046    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr" podUID="f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" containerName="payload-processing" containerID="cri-o://0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58" gracePeriod=30
Jul 28 01:59:11.487741 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.487644    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"]
Jul 28 01:59:11.489954 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.489885    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9e1b27-xg28m"]
Jul 28 01:59:11.587641 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.587615    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:59:11.613114 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.613089    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:59:11.753989 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.753928    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8rvj\" (UniqueName: \"kubernetes.io/projected/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-kube-api-access-p8rvj\") pod \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") "
Jul 28 01:59:11.754165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.754013    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zcm7\" (UniqueName: \"kubernetes.io/projected/bc6b6289-fd25-4985-82f6-d02d880611e3-kube-api-access-6zcm7\") pod \"bc6b6289-fd25-4985-82f6-d02d880611e3\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") "
Jul 28 01:59:11.754165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.754056    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-plugins-config-volume\") pod \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\" (UID: \"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b\") "
Jul 28 01:59:11.754165 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.754099    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6b6289-fd25-4985-82f6-d02d880611e3-plugins-config-volume\") pod \"bc6b6289-fd25-4985-82f6-d02d880611e3\" (UID: \"bc6b6289-fd25-4985-82f6-d02d880611e3\") "
Jul 28 01:59:11.754457 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.754433    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" (UID: "f0a0f4cb-38c8-456e-bcc7-ec85be95de7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:11.754532 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.754489    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6b6289-fd25-4985-82f6-d02d880611e3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bc6b6289-fd25-4985-82f6-d02d880611e3" (UID: "bc6b6289-fd25-4985-82f6-d02d880611e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:11.756311 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.756292    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6b6289-fd25-4985-82f6-d02d880611e3-kube-api-access-6zcm7" (OuterVolumeSpecName: "kube-api-access-6zcm7") pod "bc6b6289-fd25-4985-82f6-d02d880611e3" (UID: "bc6b6289-fd25-4985-82f6-d02d880611e3"). InnerVolumeSpecName "kube-api-access-6zcm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:11.756387 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.756326    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-kube-api-access-p8rvj" (OuterVolumeSpecName: "kube-api-access-p8rvj") pod "f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" (UID: "f0a0f4cb-38c8-456e-bcc7-ec85be95de7b"). InnerVolumeSpecName "kube-api-access-p8rvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:11.853408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.853324    2566 generic.go:356] "Generic (PLEG): container finished" podID="bc6b6289-fd25-4985-82f6-d02d880611e3" containerID="86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230" exitCode=0
Jul 28 01:59:11.853408 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.853396    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"
Jul 28 01:59:11.853617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.853411    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh" event={"ID":"bc6b6289-fd25-4985-82f6-d02d880611e3","Type":"ContainerDied","Data":"86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230"}
Jul 28 01:59:11.853617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.853505    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh" event={"ID":"bc6b6289-fd25-4985-82f6-d02d880611e3","Type":"ContainerDied","Data":"401d2e1f3de9515fc1523d688fed32e593cf79b6a16a41cb95c0510efb80798b"}
Jul 28 01:59:11.853617 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.853532    2566 scope.go:117] "RemoveContainer" containerID="86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230"
Jul 28 01:59:11.854607 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854594    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8rvj\" (UniqueName: \"kubernetes.io/projected/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-kube-api-access-p8rvj\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:11.854684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854616    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zcm7\" (UniqueName: \"kubernetes.io/projected/bc6b6289-fd25-4985-82f6-d02d880611e3-kube-api-access-6zcm7\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:11.854684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854631    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:11.854684 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854645    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc6b6289-fd25-4985-82f6-d02d880611e3-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:11.854792 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854728    2566 generic.go:356] "Generic (PLEG): container finished" podID="f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" containerID="0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58" exitCode=0
Jul 28 01:59:11.854792 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854783    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr" event={"ID":"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b","Type":"ContainerDied","Data":"0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58"}
Jul 28 01:59:11.854892 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854813    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr" event={"ID":"f0a0f4cb-38c8-456e-bcc7-ec85be95de7b","Type":"ContainerDied","Data":"5862d098a9b2d42dd0758d7a7345d01e9f19e9c9faa89125ad3b7d909b41d679"}
Jul 28 01:59:11.854947 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.854900    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"
Jul 28 01:59:11.863254 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.863239    2566 scope.go:117] "RemoveContainer" containerID="86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230"
Jul 28 01:59:11.863484 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:59:11.863467    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230\": container with ID starting with 86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230 not found: ID does not exist" containerID="86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230"
Jul 28 01:59:11.863551 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.863489    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230"} err="failed to get container status \"86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230\": rpc error: code = NotFound desc = could not find container \"86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230\": container with ID starting with 86747b3ca935ab9024039c28218258189613f8acebc34f73d4816dca21d6d230 not found: ID does not exist"
Jul 28 01:59:11.863551 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.863503    2566 scope.go:117] "RemoveContainer" containerID="0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58"
Jul 28 01:59:11.871205 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.871190    2566 scope.go:117] "RemoveContainer" containerID="0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58"
Jul 28 01:59:11.871434 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:59:11.871417    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58\": container with ID starting with 0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58 not found: ID does not exist" containerID="0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58"
Jul 28 01:59:11.871473 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.871441    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58"} err="failed to get container status \"0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58\": rpc error: code = NotFound desc = could not find container \"0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58\": container with ID starting with 0f629c6c61c2cea78b4ff8d881f8440aa5c2c008d5fdca25185e92e400595f58 not found: ID does not exist"
Jul 28 01:59:11.877518 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.877496    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"]
Jul 28 01:59:11.881699 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.881678    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9e1b27-6899c9cbd-bwpvh"]
Jul 28 01:59:11.892139 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.892112    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"]
Jul 28 01:59:11.897193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:11.897166    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-9e1b27-58ccd95dd8-gk2rr"]
Jul 28 01:59:12.560908 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:12.560876    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ae3703e-57ae-4f7e-b200-4894ccbc13d8" path="/var/lib/kubelet/pods/7ae3703e-57ae-4f7e-b200-4894ccbc13d8/volumes"
Jul 28 01:59:12.561304 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:12.561219    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc6b6289-fd25-4985-82f6-d02d880611e3" path="/var/lib/kubelet/pods/bc6b6289-fd25-4985-82f6-d02d880611e3/volumes"
Jul 28 01:59:12.561513 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:12.561500    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" path="/var/lib/kubelet/pods/f0a0f4cb-38c8-456e-bcc7-ec85be95de7b/volumes"
Jul 28 01:59:18.692712 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:18.692672    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:59:22.130823 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.130787    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"]
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131161    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae3703e-57ae-4f7e-b200-4894ccbc13d8" containerName="revoke-keys"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131174    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae3703e-57ae-4f7e-b200-4894ccbc13d8" containerName="revoke-keys"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131195    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc6b6289-fd25-4985-82f6-d02d880611e3" containerName="payload-pre-processing"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131201    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc6b6289-fd25-4985-82f6-d02d880611e3" containerName="payload-pre-processing"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131208    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f3f19df-f60f-4049-b562-4f95a2967849" containerName="registry-server"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131214    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f3f19df-f60f-4049-b562-4f95a2967849" containerName="registry-server"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131221    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" containerName="payload-processing"
Jul 28 01:59:22.131232 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131226    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" containerName="payload-processing"
Jul 28 01:59:22.131475 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131283    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a0f4cb-38c8-456e-bcc7-ec85be95de7b" containerName="payload-processing"
Jul 28 01:59:22.131475 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131292    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f3f19df-f60f-4049-b562-4f95a2967849" containerName="registry-server"
Jul 28 01:59:22.131475 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131298    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc6b6289-fd25-4985-82f6-d02d880611e3" containerName="payload-pre-processing"
Jul 28 01:59:22.131475 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.131305    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae3703e-57ae-4f7e-b200-4894ccbc13d8" containerName="revoke-keys"
Jul 28 01:59:22.136262 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.136245    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:22.138866 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.138605    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 01:59:22.139675 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.139649    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 01:59:22.142079 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.142059    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"]
Jul 28 01:59:22.231454 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.231418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d80e007-b376-4a5d-8e6b-7f06478fc388-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2\" (UID: \"4d80e007-b376-4a5d-8e6b-7f06478fc388\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:22.332201 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.332155    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d80e007-b376-4a5d-8e6b-7f06478fc388-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2\" (UID: \"4d80e007-b376-4a5d-8e6b-7f06478fc388\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:22.332843 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.332816    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d80e007-b376-4a5d-8e6b-7f06478fc388-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2\" (UID: \"4d80e007-b376-4a5d-8e6b-7f06478fc388\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:22.448759 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.448671    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:22.599661 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.599636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"]
Jul 28 01:59:22.601579 ip-10-0-131-236 kubenswrapper[2566]: W0728 01:59:22.601552    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d80e007_b376_4a5d_8e6b_7f06478fc388.slice/crio-9ce8a3bd369215f275d6b44d051ce16d08c0e00f70d575932714213240eec745 WatchSource:0}: Error finding container 9ce8a3bd369215f275d6b44d051ce16d08c0e00f70d575932714213240eec745: Status 404 returned error can't find the container with id 9ce8a3bd369215f275d6b44d051ce16d08c0e00f70d575932714213240eec745
Jul 28 01:59:22.912602 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.912563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2" event={"ID":"4d80e007-b376-4a5d-8e6b-7f06478fc388","Type":"ContainerStarted","Data":"a6157606becc890b3108052398c5f36b02a12e27c15595a62a990459c222ae40"}
Jul 28 01:59:22.912775 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.912613    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2" event={"ID":"4d80e007-b376-4a5d-8e6b-7f06478fc388","Type":"ContainerStarted","Data":"9ce8a3bd369215f275d6b44d051ce16d08c0e00f70d575932714213240eec745"}
Jul 28 01:59:22.928616 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:22.928558    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2" podStartSLOduration=0.928543498 podStartE2EDuration="928.543498ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 01:59:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 01:59:22.926889655 +0000 UTC m=+3298.938128262" watchObservedRunningTime="2026-07-28 01:59:22.928543498 +0000 UTC m=+3298.939782151"
Jul 28 01:59:23.918814 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:23.918780    2566 generic.go:356] "Generic (PLEG): container finished" podID="4d80e007-b376-4a5d-8e6b-7f06478fc388" containerID="a6157606becc890b3108052398c5f36b02a12e27c15595a62a990459c222ae40" exitCode=0
Jul 28 01:59:23.919250 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:23.918863    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2" event={"ID":"4d80e007-b376-4a5d-8e6b-7f06478fc388","Type":"ContainerDied","Data":"a6157606becc890b3108052398c5f36b02a12e27c15595a62a990459c222ae40"}
Jul 28 01:59:25.056578 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.056553    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:25.157290 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.157254    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d80e007-b376-4a5d-8e6b-7f06478fc388-maas-api-service-ca\") pod \"4d80e007-b376-4a5d-8e6b-7f06478fc388\" (UID: \"4d80e007-b376-4a5d-8e6b-7f06478fc388\") "
Jul 28 01:59:25.157608 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.157587    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d80e007-b376-4a5d-8e6b-7f06478fc388-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4d80e007-b376-4a5d-8e6b-7f06478fc388" (UID: "4d80e007-b376-4a5d-8e6b-7f06478fc388"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:25.258270 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.258187    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d80e007-b376-4a5d-8e6b-7f06478fc388-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:25.930175 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.930139    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"
Jul 28 01:59:25.930175 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.930150    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2" event={"ID":"4d80e007-b376-4a5d-8e6b-7f06478fc388","Type":"ContainerDied","Data":"9ce8a3bd369215f275d6b44d051ce16d08c0e00f70d575932714213240eec745"}
Jul 28 01:59:25.930404 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:25.930197    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ce8a3bd369215f275d6b44d051ce16d08c0e00f70d575932714213240eec745"
Jul 28 01:59:36.442045 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.442006    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 01:59:36.534321 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.534286    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"]
Jul 28 01:59:36.534565 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.534516    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq" podUID="ed3a69f7-a977-4eed-a2c3-6c57918b3c66" containerName="payload-processing" containerID="cri-o://e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e" gracePeriod=30
Jul 28 01:59:36.541110 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.541078    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"]
Jul 28 01:59:36.541380 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.541344    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr" podUID="533ab225-b26b-4d8c-91b7-d53fb13f90a1" containerName="payload-pre-processing" containerID="cri-o://eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0" gracePeriod=30
Jul 28 01:59:36.730152 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.730071    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"]
Jul 28 01:59:36.733907 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.733865    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8a33ed-jgqr2"]
Jul 28 01:59:36.797167 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.797142    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:59:36.826891 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.826867    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:59:36.867403 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.867377    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crpvb\" (UniqueName: \"kubernetes.io/projected/533ab225-b26b-4d8c-91b7-d53fb13f90a1-kube-api-access-crpvb\") pod \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") "
Jul 28 01:59:36.867591 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.867446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6wj5\" (UniqueName: \"kubernetes.io/projected/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-kube-api-access-z6wj5\") pod \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") "
Jul 28 01:59:36.867591 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.867473    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533ab225-b26b-4d8c-91b7-d53fb13f90a1-plugins-config-volume\") pod \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\" (UID: \"533ab225-b26b-4d8c-91b7-d53fb13f90a1\") "
Jul 28 01:59:36.867591 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.867528    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-plugins-config-volume\") pod \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\" (UID: \"ed3a69f7-a977-4eed-a2c3-6c57918b3c66\") "
Jul 28 01:59:36.867935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.867908    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed3a69f7-a977-4eed-a2c3-6c57918b3c66" (UID: "ed3a69f7-a977-4eed-a2c3-6c57918b3c66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:36.867935 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.867921    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533ab225-b26b-4d8c-91b7-d53fb13f90a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "533ab225-b26b-4d8c-91b7-d53fb13f90a1" (UID: "533ab225-b26b-4d8c-91b7-d53fb13f90a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 01:59:36.869749 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.869723    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-kube-api-access-z6wj5" (OuterVolumeSpecName: "kube-api-access-z6wj5") pod "ed3a69f7-a977-4eed-a2c3-6c57918b3c66" (UID: "ed3a69f7-a977-4eed-a2c3-6c57918b3c66"). InnerVolumeSpecName "kube-api-access-z6wj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:36.869851 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.869758    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533ab225-b26b-4d8c-91b7-d53fb13f90a1-kube-api-access-crpvb" (OuterVolumeSpecName: "kube-api-access-crpvb") pod "533ab225-b26b-4d8c-91b7-d53fb13f90a1" (UID: "533ab225-b26b-4d8c-91b7-d53fb13f90a1"). InnerVolumeSpecName "kube-api-access-crpvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 01:59:36.969193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.969110    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:36.969193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.969137    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crpvb\" (UniqueName: \"kubernetes.io/projected/533ab225-b26b-4d8c-91b7-d53fb13f90a1-kube-api-access-crpvb\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:36.969193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.969146    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6wj5\" (UniqueName: \"kubernetes.io/projected/ed3a69f7-a977-4eed-a2c3-6c57918b3c66-kube-api-access-z6wj5\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:36.969193 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.969155    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533ab225-b26b-4d8c-91b7-d53fb13f90a1-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 01:59:36.989248 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.989218    2566 generic.go:356] "Generic (PLEG): container finished" podID="ed3a69f7-a977-4eed-a2c3-6c57918b3c66" containerID="e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e" exitCode=0
Jul 28 01:59:36.989422 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.989268    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"
Jul 28 01:59:36.989422 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.989310    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq" event={"ID":"ed3a69f7-a977-4eed-a2c3-6c57918b3c66","Type":"ContainerDied","Data":"e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e"}
Jul 28 01:59:36.989422 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.989363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq" event={"ID":"ed3a69f7-a977-4eed-a2c3-6c57918b3c66","Type":"ContainerDied","Data":"c2b8c15777d164939e15d6b8c30ce6c16e93c2765ee19321002b58fcf856d468"}
Jul 28 01:59:36.989422 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.989390    2566 scope.go:117] "RemoveContainer" containerID="e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e"
Jul 28 01:59:36.990611 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.990592    2566 generic.go:356] "Generic (PLEG): container finished" podID="533ab225-b26b-4d8c-91b7-d53fb13f90a1" containerID="eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0" exitCode=0
Jul 28 01:59:36.990712 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.990642    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"
Jul 28 01:59:36.990712 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.990667    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr" event={"ID":"533ab225-b26b-4d8c-91b7-d53fb13f90a1","Type":"ContainerDied","Data":"eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0"}
Jul 28 01:59:36.990712 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.990709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr" event={"ID":"533ab225-b26b-4d8c-91b7-d53fb13f90a1","Type":"ContainerDied","Data":"3f0f998f971c1fc7ae66c2151af1448a565fdd5fc105296b793caf377a9a6eff"}
Jul 28 01:59:36.997996 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.997955    2566 scope.go:117] "RemoveContainer" containerID="e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e"
Jul 28 01:59:36.998285 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:59:36.998264    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e\": container with ID starting with e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e not found: ID does not exist" containerID="e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e"
Jul 28 01:59:36.998359 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.998296    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e"} err="failed to get container status \"e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e\": rpc error: code = NotFound desc = could not find container \"e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e\": container with ID starting with e334f1d63064b57df38ca67e5d604a893c2cb0a950982c61cb1ed1e7e416b15e not found: ID does not exist"
Jul 28 01:59:36.998359 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:36.998318    2566 scope.go:117] "RemoveContainer" containerID="eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0"
Jul 28 01:59:37.006649 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:37.006632    2566 scope.go:117] "RemoveContainer" containerID="eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0"
Jul 28 01:59:37.006902 ip-10-0-131-236 kubenswrapper[2566]: E0728 01:59:37.006885    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0\": container with ID starting with eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0 not found: ID does not exist" containerID="eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0"
Jul 28 01:59:37.006945 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:37.006909    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0"} err="failed to get container status \"eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0\": rpc error: code = NotFound desc = could not find container \"eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0\": container with ID starting with eb045c4fb1caaac1fcdeafbbdcdc71a9be9b77c0a39372176fc6e4255643d6d0 not found: ID does not exist"
Jul 28 01:59:37.016769 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:37.016745    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"]
Jul 28 01:59:37.022952 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:37.022927    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-8a33ed-6746bcb4d6-qn4wq"]
Jul 28 01:59:37.037050 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:37.037021    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"]
Jul 28 01:59:37.043493 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:37.043471    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8a33ed-766f8965d4-t7rkr"]
Jul 28 01:59:38.560961 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:38.560927    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d80e007-b376-4a5d-8e6b-7f06478fc388" path="/var/lib/kubelet/pods/4d80e007-b376-4a5d-8e6b-7f06478fc388/volumes"
Jul 28 01:59:38.561332 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:38.561260    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533ab225-b26b-4d8c-91b7-d53fb13f90a1" path="/var/lib/kubelet/pods/533ab225-b26b-4d8c-91b7-d53fb13f90a1/volumes"
Jul 28 01:59:38.561552 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:38.561540    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3a69f7-a977-4eed-a2c3-6c57918b3c66" path="/var/lib/kubelet/pods/ed3a69f7-a977-4eed-a2c3-6c57918b3c66/volumes"
Jul 28 01:59:46.532131 ip-10-0-131-236 kubenswrapper[2566]: I0728 01:59:46.532037    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 02:00:00.138220 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138180    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"]
Jul 28 02:00:00.138742 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138723    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533ab225-b26b-4d8c-91b7-d53fb13f90a1" containerName="payload-pre-processing"
Jul 28 02:00:00.138787 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138746    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="533ab225-b26b-4d8c-91b7-d53fb13f90a1" containerName="payload-pre-processing"
Jul 28 02:00:00.138787 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138763    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d80e007-b376-4a5d-8e6b-7f06478fc388" containerName="revoke-keys"
Jul 28 02:00:00.138787 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138772    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d80e007-b376-4a5d-8e6b-7f06478fc388" containerName="revoke-keys"
Jul 28 02:00:00.138879 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138789    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3a69f7-a977-4eed-a2c3-6c57918b3c66" containerName="payload-processing"
Jul 28 02:00:00.138879 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138797    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3a69f7-a977-4eed-a2c3-6c57918b3c66" containerName="payload-processing"
Jul 28 02:00:00.138879 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138873    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d80e007-b376-4a5d-8e6b-7f06478fc388" containerName="revoke-keys"
Jul 28 02:00:00.138987 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138883    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="533ab225-b26b-4d8c-91b7-d53fb13f90a1" containerName="payload-pre-processing"
Jul 28 02:00:00.138987 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.138892    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3a69f7-a977-4eed-a2c3-6c57918b3c66" containerName="payload-processing"
Jul 28 02:00:00.149694 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.144136    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"]
Jul 28 02:00:00.149694 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.149397    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"]
Jul 28 02:00:00.149694 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.149677    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"
Jul 28 02:00:00.149998 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.149894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"
Jul 28 02:00:00.153087 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.153066    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 02:00:00.153223 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.153140    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"]
Jul 28 02:00:00.153291 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.153273    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"
Jul 28 02:00:00.156179 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.156163    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"
Jul 28 02:00:00.162061 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.162041    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"
Jul 28 02:00:00.162381 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.162360    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"]
Jul 28 02:00:00.164933 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.164911    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"]
Jul 28 02:00:00.167194 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.167171    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"]
Jul 28 02:00:00.177904 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.177880    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"
Jul 28 02:00:00.182960 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.182928    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"]
Jul 28 02:00:00.214997 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.214850    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"
Jul 28 02:00:00.222695 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.222667    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"
Jul 28 02:00:00.364950 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.364873    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"]
Jul 28 02:00:00.368169 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:00:00.368139    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c85757b_a0a2_4909_9e7b_032488a92c1e.slice/crio-ad2c9801bb741fe5d55b6914541834157c2fd2418b710beecef6e472fc9b7e0e WatchSource:0}: Error finding container ad2c9801bb741fe5d55b6914541834157c2fd2418b710beecef6e472fc9b7e0e: Status 404 returned error can't find the container with id ad2c9801bb741fe5d55b6914541834157c2fd2418b710beecef6e472fc9b7e0e
Jul 28 02:00:00.423950 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.423926    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"]
Jul 28 02:00:00.426018 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:00:00.425992    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60407a9b_d80e_4f7d_aa93_1dbad04a51bd.slice/crio-e8f929199963f956507101a27e958b2705c3404116f90a44ee00de86643f3c46 WatchSource:0}: Error finding container e8f929199963f956507101a27e958b2705c3404116f90a44ee00de86643f3c46: Status 404 returned error can't find the container with id e8f929199963f956507101a27e958b2705c3404116f90a44ee00de86643f3c46
Jul 28 02:00:00.584382 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.584346    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"]
Jul 28 02:00:00.586489 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:00:00.586453    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf096d179_9fb9_467b_b617_19ff05bc46fa.slice/crio-291bd85abc9c6bbeb8eb74bc38d6e500f2fcabcfed8bc1a381cef6fb9ade0d29 WatchSource:0}: Error finding container 291bd85abc9c6bbeb8eb74bc38d6e500f2fcabcfed8bc1a381cef6fb9ade0d29: Status 404 returned error can't find the container with id 291bd85abc9c6bbeb8eb74bc38d6e500f2fcabcfed8bc1a381cef6fb9ade0d29
Jul 28 02:00:00.608497 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:00.608405    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"]
Jul 28 02:00:00.623218 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:00:00.623187    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b32c449_ff1c_458d_a6de_f6966fffe4b6.slice/crio-e3b4e2e6138868dd7b72eb47c0f18013f636a774ca86262c6b87a4bf6ff417b9 WatchSource:0}: Error finding container e3b4e2e6138868dd7b72eb47c0f18013f636a774ca86262c6b87a4bf6ff417b9: Status 404 returned error can't find the container with id e3b4e2e6138868dd7b72eb47c0f18013f636a774ca86262c6b87a4bf6ff417b9
Jul 28 02:00:01.110425 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.110391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb" event={"ID":"7b32c449-ff1c-458d-a6de-f6966fffe4b6","Type":"ContainerStarted","Data":"4d411dc505ef24debfd6253c307feb77eca5d788581e7692f4bbd39eb9175e30"}
Jul 28 02:00:01.110615 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.110446    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb" event={"ID":"7b32c449-ff1c-458d-a6de-f6966fffe4b6","Type":"ContainerStarted","Data":"e3b4e2e6138868dd7b72eb47c0f18013f636a774ca86262c6b87a4bf6ff417b9"}
Jul 28 02:00:01.111805 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.111779    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj" event={"ID":"60407a9b-d80e-4f7d-aa93-1dbad04a51bd","Type":"ContainerStarted","Data":"d4a95912047c6f27de3ae732874a0663339747582a3522ac8373e0132b681b4c"}
Jul 28 02:00:01.111937 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.111829    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj" event={"ID":"60407a9b-d80e-4f7d-aa93-1dbad04a51bd","Type":"ContainerStarted","Data":"e8f929199963f956507101a27e958b2705c3404116f90a44ee00de86643f3c46"}
Jul 28 02:00:01.113114 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.113093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t" event={"ID":"3c85757b-a0a2-4909-9e7b-032488a92c1e","Type":"ContainerStarted","Data":"5b0fe6923ddceeee3df09508ee93913b02e681140ee82e4745956000f369919a"}
Jul 28 02:00:01.113231 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.113135    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t" event={"ID":"3c85757b-a0a2-4909-9e7b-032488a92c1e","Type":"ContainerStarted","Data":"ad2c9801bb741fe5d55b6914541834157c2fd2418b710beecef6e472fc9b7e0e"}
Jul 28 02:00:01.114473 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.114450    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7" event={"ID":"f096d179-9fb9-467b-b617-19ff05bc46fa","Type":"ContainerStarted","Data":"28e02b5f8371fe4eac3a3042250ef9720772bbb3432001b55facc53d1177a1e5"}
Jul 28 02:00:01.114572 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.114493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7" event={"ID":"f096d179-9fb9-467b-b617-19ff05bc46fa","Type":"ContainerStarted","Data":"291bd85abc9c6bbeb8eb74bc38d6e500f2fcabcfed8bc1a381cef6fb9ade0d29"}
Jul 28 02:00:01.126756 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.126707    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb" podStartSLOduration=1.126693768 podStartE2EDuration="1.126693768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:00:01.125339615 +0000 UTC m=+3337.136578236" watchObservedRunningTime="2026-07-28 02:00:01.126693768 +0000 UTC m=+3337.137932367"
Jul 28 02:00:01.140072 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.140021    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7" podStartSLOduration=1.139957756 podStartE2EDuration="1.139957756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:00:01.139929881 +0000 UTC m=+3337.151168488" watchObservedRunningTime="2026-07-28 02:00:01.139957756 +0000 UTC m=+3337.151196364"
Jul 28 02:00:01.161624 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.161577    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj" podStartSLOduration=1.161562788 podStartE2EDuration="1.161562788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:00:01.15990507 +0000 UTC m=+3337.171143679" watchObservedRunningTime="2026-07-28 02:00:01.161562788 +0000 UTC m=+3337.172801442"
Jul 28 02:00:01.185875 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:01.185815    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t" podStartSLOduration=1.185800993 podStartE2EDuration="1.185800993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 02:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:00:01.184072514 +0000 UTC m=+3337.195311119" watchObservedRunningTime="2026-07-28 02:00:01.185800993 +0000 UTC m=+3337.197039599"
Jul 28 02:00:02.121226 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.121193    2566 generic.go:356] "Generic (PLEG): container finished" podID="60407a9b-d80e-4f7d-aa93-1dbad04a51bd" containerID="d4a95912047c6f27de3ae732874a0663339747582a3522ac8373e0132b681b4c" exitCode=0
Jul 28 02:00:02.121429 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.121283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj" event={"ID":"60407a9b-d80e-4f7d-aa93-1dbad04a51bd","Type":"ContainerDied","Data":"d4a95912047c6f27de3ae732874a0663339747582a3522ac8373e0132b681b4c"}
Jul 28 02:00:02.122917 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.122889    2566 generic.go:356] "Generic (PLEG): container finished" podID="3c85757b-a0a2-4909-9e7b-032488a92c1e" containerID="5b0fe6923ddceeee3df09508ee93913b02e681140ee82e4745956000f369919a" exitCode=0
Jul 28 02:00:02.123044 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.122990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t" event={"ID":"3c85757b-a0a2-4909-9e7b-032488a92c1e","Type":"ContainerDied","Data":"5b0fe6923ddceeee3df09508ee93913b02e681140ee82e4745956000f369919a"}
Jul 28 02:00:02.124519 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.124489    2566 generic.go:356] "Generic (PLEG): container finished" podID="f096d179-9fb9-467b-b617-19ff05bc46fa" containerID="28e02b5f8371fe4eac3a3042250ef9720772bbb3432001b55facc53d1177a1e5" exitCode=0
Jul 28 02:00:02.124630 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.124564    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7" event={"ID":"f096d179-9fb9-467b-b617-19ff05bc46fa","Type":"ContainerDied","Data":"28e02b5f8371fe4eac3a3042250ef9720772bbb3432001b55facc53d1177a1e5"}
Jul 28 02:00:02.125953 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.125926    2566 generic.go:356] "Generic (PLEG): container finished" podID="7b32c449-ff1c-458d-a6de-f6966fffe4b6" containerID="4d411dc505ef24debfd6253c307feb77eca5d788581e7692f4bbd39eb9175e30" exitCode=0
Jul 28 02:00:02.126051 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:02.125996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb" event={"ID":"7b32c449-ff1c-458d-a6de-f6966fffe4b6","Type":"ContainerDied","Data":"4d411dc505ef24debfd6253c307feb77eca5d788581e7692f4bbd39eb9175e30"}
Jul 28 02:00:03.285837 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:03.285807    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"
Jul 28 02:00:03.343405 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:03.343380    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"
Jul 28 02:00:03.346744 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:03.346722    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"
Jul 28 02:00:03.350090 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:03.350068    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"
Jul 28 02:00:04.138208 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.138171    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"
Jul 28 02:00:04.138405 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.138204    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb" event={"ID":"7b32c449-ff1c-458d-a6de-f6966fffe4b6","Type":"ContainerDied","Data":"e3b4e2e6138868dd7b72eb47c0f18013f636a774ca86262c6b87a4bf6ff417b9"}
Jul 28 02:00:04.138405 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.138253    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b4e2e6138868dd7b72eb47c0f18013f636a774ca86262c6b87a4bf6ff417b9"
Jul 28 02:00:04.139434 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.139417    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"
Jul 28 02:00:04.139434 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.139429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj" event={"ID":"60407a9b-d80e-4f7d-aa93-1dbad04a51bd","Type":"ContainerDied","Data":"e8f929199963f956507101a27e958b2705c3404116f90a44ee00de86643f3c46"}
Jul 28 02:00:04.139606 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.139459    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e8f929199963f956507101a27e958b2705c3404116f90a44ee00de86643f3c46"
Jul 28 02:00:04.140697 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.140665    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"
Jul 28 02:00:04.140697 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.140677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t" event={"ID":"3c85757b-a0a2-4909-9e7b-032488a92c1e","Type":"ContainerDied","Data":"ad2c9801bb741fe5d55b6914541834157c2fd2418b710beecef6e472fc9b7e0e"}
Jul 28 02:00:04.140851 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.140716    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad2c9801bb741fe5d55b6914541834157c2fd2418b710beecef6e472fc9b7e0e"
Jul 28 02:00:04.141960 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.141935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7" event={"ID":"f096d179-9fb9-467b-b617-19ff05bc46fa","Type":"ContainerDied","Data":"291bd85abc9c6bbeb8eb74bc38d6e500f2fcabcfed8bc1a381cef6fb9ade0d29"}
Jul 28 02:00:04.142061 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.141998    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753400-55dn7"
Jul 28 02:00:04.142061 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:04.142002    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="291bd85abc9c6bbeb8eb74bc38d6e500f2fcabcfed8bc1a381cef6fb9ade0d29"
Jul 28 02:00:40.685490 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685454    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"]
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685807    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60407a9b-d80e-4f7d-aa93-1dbad04a51bd" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685818    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="60407a9b-d80e-4f7d-aa93-1dbad04a51bd" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685834    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f096d179-9fb9-467b-b617-19ff05bc46fa" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685839    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f096d179-9fb9-467b-b617-19ff05bc46fa" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685849    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b32c449-ff1c-458d-a6de-f6966fffe4b6" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685855    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b32c449-ff1c-458d-a6de-f6966fffe4b6" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685865    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c85757b-a0a2-4909-9e7b-032488a92c1e" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685870    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c85757b-a0a2-4909-9e7b-032488a92c1e" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685924    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="60407a9b-d80e-4f7d-aa93-1dbad04a51bd" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685934    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b32c449-ff1c-458d-a6de-f6966fffe4b6" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685942    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c85757b-a0a2-4909-9e7b-032488a92c1e" containerName="cleanup"
Jul 28 02:00:40.686049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.685947    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f096d179-9fb9-467b-b617-19ff05bc46fa" containerName="cleanup"
Jul 28 02:00:40.689034 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.689017    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:40.691410 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.691385    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 02:00:40.691512 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.691393    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 02:00:40.697073 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.697046    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"]
Jul 28 02:00:40.820874 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.820826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f008c0-18d3-48b8-a742-00ad546e19d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7\" (UID: \"72f008c0-18d3-48b8-a742-00ad546e19d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:40.921355 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.921321    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f008c0-18d3-48b8-a742-00ad546e19d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7\" (UID: \"72f008c0-18d3-48b8-a742-00ad546e19d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:40.921929 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:40.921911    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f008c0-18d3-48b8-a742-00ad546e19d9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7\" (UID: \"72f008c0-18d3-48b8-a742-00ad546e19d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:41.001047 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:41.000928    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:41.143779 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:41.143757    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"]
Jul 28 02:00:41.146060 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:00:41.146034    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72f008c0_18d3_48b8_a742_00ad546e19d9.slice/crio-52167aa1f7fc5fcf52444f25fcfed90b11029486b315ff8f42700641221187fe WatchSource:0}: Error finding container 52167aa1f7fc5fcf52444f25fcfed90b11029486b315ff8f42700641221187fe: Status 404 returned error can't find the container with id 52167aa1f7fc5fcf52444f25fcfed90b11029486b315ff8f42700641221187fe
Jul 28 02:00:41.338071 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:41.338039    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7" event={"ID":"72f008c0-18d3-48b8-a742-00ad546e19d9","Type":"ContainerStarted","Data":"52167aa1f7fc5fcf52444f25fcfed90b11029486b315ff8f42700641221187fe"}
Jul 28 02:00:42.345831 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:42.345790    2566 generic.go:356] "Generic (PLEG): container finished" podID="72f008c0-18d3-48b8-a742-00ad546e19d9" containerID="d86120e69d21f233ae2fcacf495da459b098e4ebc7d5d67646a5e31720afd634" exitCode=0
Jul 28 02:00:42.346256 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:42.345880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7" event={"ID":"72f008c0-18d3-48b8-a742-00ad546e19d9","Type":"ContainerDied","Data":"d86120e69d21f233ae2fcacf495da459b098e4ebc7d5d67646a5e31720afd634"}
Jul 28 02:00:43.492226 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:43.492202    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:43.645940 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:43.645852    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f008c0-18d3-48b8-a742-00ad546e19d9-maas-api-service-ca\") pod \"72f008c0-18d3-48b8-a742-00ad546e19d9\" (UID: \"72f008c0-18d3-48b8-a742-00ad546e19d9\") "
Jul 28 02:00:43.646255 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:43.646229    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f008c0-18d3-48b8-a742-00ad546e19d9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72f008c0-18d3-48b8-a742-00ad546e19d9" (UID: "72f008c0-18d3-48b8-a742-00ad546e19d9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:00:43.747379 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:43.747350    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72f008c0-18d3-48b8-a742-00ad546e19d9-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:00:44.356885 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:44.356848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7" event={"ID":"72f008c0-18d3-48b8-a742-00ad546e19d9","Type":"ContainerDied","Data":"52167aa1f7fc5fcf52444f25fcfed90b11029486b315ff8f42700641221187fe"}
Jul 28 02:00:44.356885 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:44.356889    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"
Jul 28 02:00:44.357124 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:44.356905    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52167aa1f7fc5fcf52444f25fcfed90b11029486b315ff8f42700641221187fe"
Jul 28 02:00:47.671643 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.671603    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"]
Jul 28 02:00:47.677007 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.676980    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-b-c430e4-29753400-d4hlb"]
Jul 28 02:00:47.706363 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.706331    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"]
Jul 28 02:00:47.706603 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.706580    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6" podUID="c516e29f-3f85-4436-957d-fcfd64c42582" containerName="payload-processing" containerID="cri-o://16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927" gracePeriod=30
Jul 28 02:00:47.716755 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.716723    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"]
Jul 28 02:00:47.717008 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.716962    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd" podUID="775c435b-ddde-4d68-9c4c-8019c542191c" containerName="payload-pre-processing" containerID="cri-o://609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720" gracePeriod=30
Jul 28 02:00:47.879029 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.878985    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"]
Jul 28 02:00:47.885375 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.885333    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-c430e4-2dld7"]
Jul 28 02:00:47.999102 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:47.999070    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 02:00:48.002088 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.002063    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 02:00:48.083278 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.083245    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775c435b-ddde-4d68-9c4c-8019c542191c-plugins-config-volume\") pod \"775c435b-ddde-4d68-9c4c-8019c542191c\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") "
Jul 28 02:00:48.083464 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.083339    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qz52\" (UniqueName: \"kubernetes.io/projected/775c435b-ddde-4d68-9c4c-8019c542191c-kube-api-access-4qz52\") pod \"775c435b-ddde-4d68-9c4c-8019c542191c\" (UID: \"775c435b-ddde-4d68-9c4c-8019c542191c\") "
Jul 28 02:00:48.083464 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.083358    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516e29f-3f85-4436-957d-fcfd64c42582-plugins-config-volume\") pod \"c516e29f-3f85-4436-957d-fcfd64c42582\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") "
Jul 28 02:00:48.083464 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.083375    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtv9f\" (UniqueName: \"kubernetes.io/projected/c516e29f-3f85-4436-957d-fcfd64c42582-kube-api-access-xtv9f\") pod \"c516e29f-3f85-4436-957d-fcfd64c42582\" (UID: \"c516e29f-3f85-4436-957d-fcfd64c42582\") "
Jul 28 02:00:48.083687 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.083657    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/775c435b-ddde-4d68-9c4c-8019c542191c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "775c435b-ddde-4d68-9c4c-8019c542191c" (UID: "775c435b-ddde-4d68-9c4c-8019c542191c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:00:48.083759 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.083724    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c516e29f-3f85-4436-957d-fcfd64c42582-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c516e29f-3f85-4436-957d-fcfd64c42582" (UID: "c516e29f-3f85-4436-957d-fcfd64c42582"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:00:48.085557 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.085525    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775c435b-ddde-4d68-9c4c-8019c542191c-kube-api-access-4qz52" (OuterVolumeSpecName: "kube-api-access-4qz52") pod "775c435b-ddde-4d68-9c4c-8019c542191c" (UID: "775c435b-ddde-4d68-9c4c-8019c542191c"). InnerVolumeSpecName "kube-api-access-4qz52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:00:48.085661 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.085626    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c516e29f-3f85-4436-957d-fcfd64c42582-kube-api-access-xtv9f" (OuterVolumeSpecName: "kube-api-access-xtv9f") pod "c516e29f-3f85-4436-957d-fcfd64c42582" (UID: "c516e29f-3f85-4436-957d-fcfd64c42582"). InnerVolumeSpecName "kube-api-access-xtv9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:00:48.184718 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.184675    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qz52\" (UniqueName: \"kubernetes.io/projected/775c435b-ddde-4d68-9c4c-8019c542191c-kube-api-access-4qz52\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.184718 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.184710    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c516e29f-3f85-4436-957d-fcfd64c42582-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.184718 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.184719    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtv9f\" (UniqueName: \"kubernetes.io/projected/c516e29f-3f85-4436-957d-fcfd64c42582-kube-api-access-xtv9f\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.184718 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.184728    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775c435b-ddde-4d68-9c4c-8019c542191c-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:00:48.378694 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.378661    2566 generic.go:356] "Generic (PLEG): container finished" podID="c516e29f-3f85-4436-957d-fcfd64c42582" containerID="16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927" exitCode=0
Jul 28 02:00:48.378876 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.378707    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"
Jul 28 02:00:48.378876 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.378738    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6" event={"ID":"c516e29f-3f85-4436-957d-fcfd64c42582","Type":"ContainerDied","Data":"16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927"}
Jul 28 02:00:48.378876 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.378790    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6" event={"ID":"c516e29f-3f85-4436-957d-fcfd64c42582","Type":"ContainerDied","Data":"e3a0d8053fb61bcec0cb3d1627e77221eed18c9edc0aef33613ae782d66a1713"}
Jul 28 02:00:48.378876 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.378819    2566 scope.go:117] "RemoveContainer" containerID="16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927"
Jul 28 02:00:48.380109 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.380088    2566 generic.go:356] "Generic (PLEG): container finished" podID="775c435b-ddde-4d68-9c4c-8019c542191c" containerID="609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720" exitCode=0
Jul 28 02:00:48.380199 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.380142    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"
Jul 28 02:00:48.380199 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.380153    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd" event={"ID":"775c435b-ddde-4d68-9c4c-8019c542191c","Type":"ContainerDied","Data":"609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720"}
Jul 28 02:00:48.380199 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.380182    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd" event={"ID":"775c435b-ddde-4d68-9c4c-8019c542191c","Type":"ContainerDied","Data":"6085bb9cfb12800c866c8cfc83b9678cb399a886d96317a262602141a64ef96e"}
Jul 28 02:00:48.388099 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.388082    2566 scope.go:117] "RemoveContainer" containerID="16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927"
Jul 28 02:00:48.388378 ip-10-0-131-236 kubenswrapper[2566]: E0728 02:00:48.388357    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927\": container with ID starting with 16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927 not found: ID does not exist" containerID="16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927"
Jul 28 02:00:48.388442 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.388390    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927"} err="failed to get container status \"16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927\": rpc error: code = NotFound desc = could not find container \"16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927\": container with ID starting with 16fe91e5482f0cfbbee5ea284f845f55ae140d8444c03ab659c33a576bbb5927 not found: ID does not exist"
Jul 28 02:00:48.388442 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.388409    2566 scope.go:117] "RemoveContainer" containerID="609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720"
Jul 28 02:00:48.396432 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.396408    2566 scope.go:117] "RemoveContainer" containerID="609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720"
Jul 28 02:00:48.396684 ip-10-0-131-236 kubenswrapper[2566]: E0728 02:00:48.396664    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720\": container with ID starting with 609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720 not found: ID does not exist" containerID="609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720"
Jul 28 02:00:48.396743 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.396695    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720"} err="failed to get container status \"609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720\": rpc error: code = NotFound desc = could not find container \"609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720\": container with ID starting with 609ed1f3039432e2b75e2c924454d7d63519974c325bb996ea3c760ad806c720 not found: ID does not exist"
Jul 28 02:00:48.405125 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.405105    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"]
Jul 28 02:00:48.407749 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.407729    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c430e4-7ff7c75d44-dm2vd"]
Jul 28 02:00:48.419157 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.419131    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"]
Jul 28 02:00:48.420934 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.420914    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c430e4-c9dd7cd6b-2rxt6"]
Jul 28 02:00:48.562111 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.562081    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f008c0-18d3-48b8-a742-00ad546e19d9" path="/var/lib/kubelet/pods/72f008c0-18d3-48b8-a742-00ad546e19d9/volumes"
Jul 28 02:00:48.562512 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.562492    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="775c435b-ddde-4d68-9c4c-8019c542191c" path="/var/lib/kubelet/pods/775c435b-ddde-4d68-9c4c-8019c542191c/volumes"
Jul 28 02:00:48.562789 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.562772    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b32c449-ff1c-458d-a6de-f6966fffe4b6" path="/var/lib/kubelet/pods/7b32c449-ff1c-458d-a6de-f6966fffe4b6/volumes"
Jul 28 02:00:48.563121 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:48.563105    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c516e29f-3f85-4436-957d-fcfd64c42582" path="/var/lib/kubelet/pods/c516e29f-3f85-4436-957d-fcfd64c42582/volumes"
Jul 28 02:00:50.932270 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:50.932235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 02:00:55.127469 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127421    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"]
Jul 28 02:00:55.127893 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127830    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f008c0-18d3-48b8-a742-00ad546e19d9" containerName="revoke-keys"
Jul 28 02:00:55.127893 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127847    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f008c0-18d3-48b8-a742-00ad546e19d9" containerName="revoke-keys"
Jul 28 02:00:55.127893 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127867    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="775c435b-ddde-4d68-9c4c-8019c542191c" containerName="payload-pre-processing"
Jul 28 02:00:55.127893 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127876    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="775c435b-ddde-4d68-9c4c-8019c542191c" containerName="payload-pre-processing"
Jul 28 02:00:55.127893 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127890    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c516e29f-3f85-4436-957d-fcfd64c42582" containerName="payload-processing"
Jul 28 02:00:55.127893 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127896    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c516e29f-3f85-4436-957d-fcfd64c42582" containerName="payload-processing"
Jul 28 02:00:55.128154 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.127998    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f008c0-18d3-48b8-a742-00ad546e19d9" containerName="revoke-keys"
Jul 28 02:00:55.128154 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.128013    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="775c435b-ddde-4d68-9c4c-8019c542191c" containerName="payload-pre-processing"
Jul 28 02:00:55.128154 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.128023    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c516e29f-3f85-4436-957d-fcfd64c42582" containerName="payload-processing"
Jul 28 02:00:55.132398 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.132381    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:55.134847 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.134828    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 02:00:55.134942 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.134850    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 02:00:55.138995 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.138948    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"]
Jul 28 02:00:55.250458 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.250424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/458fc273-7327-4971-9a99-022b941e37c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6\" (UID: \"458fc273-7327-4971-9a99-022b941e37c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:55.350908 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.350875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/458fc273-7327-4971-9a99-022b941e37c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6\" (UID: \"458fc273-7327-4971-9a99-022b941e37c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:55.351538 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.351520    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/458fc273-7327-4971-9a99-022b941e37c7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6\" (UID: \"458fc273-7327-4971-9a99-022b941e37c7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:55.443542 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.443455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:55.579239 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:55.579210    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"]
Jul 28 02:00:55.581957 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:00:55.581929    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458fc273_7327_4971_9a99_022b941e37c7.slice/crio-7e7295e52885c72cb705a111e6cec4d4f390699f034556d6dff17275a52f10cf WatchSource:0}: Error finding container 7e7295e52885c72cb705a111e6cec4d4f390699f034556d6dff17275a52f10cf: Status 404 returned error can't find the container with id 7e7295e52885c72cb705a111e6cec4d4f390699f034556d6dff17275a52f10cf
Jul 28 02:00:56.424137 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:56.424042    2566 generic.go:356] "Generic (PLEG): container finished" podID="458fc273-7327-4971-9a99-022b941e37c7" containerID="2cf97887276c13f0f62c79987020b7bf9accd55bfa1e68aa8a443e876074abf0" exitCode=0
Jul 28 02:00:56.424520 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:56.424132    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6" event={"ID":"458fc273-7327-4971-9a99-022b941e37c7","Type":"ContainerDied","Data":"2cf97887276c13f0f62c79987020b7bf9accd55bfa1e68aa8a443e876074abf0"}
Jul 28 02:00:56.424520 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:56.424188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6" event={"ID":"458fc273-7327-4971-9a99-022b941e37c7","Type":"ContainerStarted","Data":"7e7295e52885c72cb705a111e6cec4d4f390699f034556d6dff17275a52f10cf"}
Jul 28 02:00:57.565641 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:57.565618    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:57.672405 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:57.672375    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/458fc273-7327-4971-9a99-022b941e37c7-maas-api-service-ca\") pod \"458fc273-7327-4971-9a99-022b941e37c7\" (UID: \"458fc273-7327-4971-9a99-022b941e37c7\") "
Jul 28 02:00:57.672750 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:57.672725    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458fc273-7327-4971-9a99-022b941e37c7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "458fc273-7327-4971-9a99-022b941e37c7" (UID: "458fc273-7327-4971-9a99-022b941e37c7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:00:57.773828 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:57.773739    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/458fc273-7327-4971-9a99-022b941e37c7-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:00:58.435536 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:58.435506    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"
Jul 28 02:00:58.435712 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:58.435544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6" event={"ID":"458fc273-7327-4971-9a99-022b941e37c7","Type":"ContainerDied","Data":"7e7295e52885c72cb705a111e6cec4d4f390699f034556d6dff17275a52f10cf"}
Jul 28 02:00:58.435712 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:58.435618    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e7295e52885c72cb705a111e6cec4d4f390699f034556d6dff17275a52f10cf"
Jul 28 02:00:59.953311 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:59.953273    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"]
Jul 28 02:00:59.956069 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:59.956040    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-0337e8-29753400-bpphj"]
Jul 28 02:00:59.985687 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:59.985653    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"]
Jul 28 02:00:59.985977 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:59.985908    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9" podUID="cef226bd-348d-40a3-88f9-b8a124df15b1" containerName="payload-processing" containerID="cri-o://37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b" gracePeriod=30
Jul 28 02:00:59.989449 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:59.989423    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"]
Jul 28 02:00:59.989819 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:00:59.989792    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6" podUID="2f720199-3725-4252-8280-a5122beb7590" containerName="payload-pre-processing" containerID="cri-o://a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93" gracePeriod=30
Jul 28 02:01:00.147620 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.147581    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"]
Jul 28 02:01:00.149118 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.149088    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-0337e8-9dkg6"]
Jul 28 02:01:00.277941 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.277917    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 02:01:00.280606 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.280588    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 02:01:00.291353 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.291317    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfg5h\" (UniqueName: \"kubernetes.io/projected/2f720199-3725-4252-8280-a5122beb7590-kube-api-access-hfg5h\") pod \"2f720199-3725-4252-8280-a5122beb7590\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") "
Jul 28 02:01:00.291481 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.291363    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f720199-3725-4252-8280-a5122beb7590-plugins-config-volume\") pod \"2f720199-3725-4252-8280-a5122beb7590\" (UID: \"2f720199-3725-4252-8280-a5122beb7590\") "
Jul 28 02:01:00.291481 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.291392    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hdn6\" (UniqueName: \"kubernetes.io/projected/cef226bd-348d-40a3-88f9-b8a124df15b1-kube-api-access-9hdn6\") pod \"cef226bd-348d-40a3-88f9-b8a124df15b1\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") "
Jul 28 02:01:00.291481 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.291431    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cef226bd-348d-40a3-88f9-b8a124df15b1-plugins-config-volume\") pod \"cef226bd-348d-40a3-88f9-b8a124df15b1\" (UID: \"cef226bd-348d-40a3-88f9-b8a124df15b1\") "
Jul 28 02:01:00.291840 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.291815    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef226bd-348d-40a3-88f9-b8a124df15b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cef226bd-348d-40a3-88f9-b8a124df15b1" (UID: "cef226bd-348d-40a3-88f9-b8a124df15b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:00.291892 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.291812    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f720199-3725-4252-8280-a5122beb7590-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2f720199-3725-4252-8280-a5122beb7590" (UID: "2f720199-3725-4252-8280-a5122beb7590"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:00.293998 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.293935    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef226bd-348d-40a3-88f9-b8a124df15b1-kube-api-access-9hdn6" (OuterVolumeSpecName: "kube-api-access-9hdn6") pod "cef226bd-348d-40a3-88f9-b8a124df15b1" (UID: "cef226bd-348d-40a3-88f9-b8a124df15b1"). InnerVolumeSpecName "kube-api-access-9hdn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:01:00.294120 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.294032    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f720199-3725-4252-8280-a5122beb7590-kube-api-access-hfg5h" (OuterVolumeSpecName: "kube-api-access-hfg5h") pod "2f720199-3725-4252-8280-a5122beb7590" (UID: "2f720199-3725-4252-8280-a5122beb7590"). InnerVolumeSpecName "kube-api-access-hfg5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:01:00.392237 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.392188    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfg5h\" (UniqueName: \"kubernetes.io/projected/2f720199-3725-4252-8280-a5122beb7590-kube-api-access-hfg5h\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.392237 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.392231    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2f720199-3725-4252-8280-a5122beb7590-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.392237 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.392240    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hdn6\" (UniqueName: \"kubernetes.io/projected/cef226bd-348d-40a3-88f9-b8a124df15b1-kube-api-access-9hdn6\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.392237 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.392250    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cef226bd-348d-40a3-88f9-b8a124df15b1-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:00.449076 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.449036    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f720199-3725-4252-8280-a5122beb7590" containerID="a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93" exitCode=0
Jul 28 02:01:00.449259 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.449086    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"
Jul 28 02:01:00.449259 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.449115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6" event={"ID":"2f720199-3725-4252-8280-a5122beb7590","Type":"ContainerDied","Data":"a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93"}
Jul 28 02:01:00.449259 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.449166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6" event={"ID":"2f720199-3725-4252-8280-a5122beb7590","Type":"ContainerDied","Data":"d9809c1ed21e9be2cdac0fd2360c10fcbf75c119f64484f9e9982b8b0f62f975"}
Jul 28 02:01:00.449259 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.449188    2566 scope.go:117] "RemoveContainer" containerID="a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93"
Jul 28 02:01:00.450530 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.450509    2566 generic.go:356] "Generic (PLEG): container finished" podID="cef226bd-348d-40a3-88f9-b8a124df15b1" containerID="37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b" exitCode=0
Jul 28 02:01:00.450666 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.450561    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"
Jul 28 02:01:00.450666 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.450601    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9" event={"ID":"cef226bd-348d-40a3-88f9-b8a124df15b1","Type":"ContainerDied","Data":"37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b"}
Jul 28 02:01:00.450832 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.450788    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9" event={"ID":"cef226bd-348d-40a3-88f9-b8a124df15b1","Type":"ContainerDied","Data":"316e89b0f1969fac74fe9659c0e9643ff7e39b958fee387c687f02a38ead0a62"}
Jul 28 02:01:00.457980 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.457949    2566 scope.go:117] "RemoveContainer" containerID="a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93"
Jul 28 02:01:00.458258 ip-10-0-131-236 kubenswrapper[2566]: E0728 02:01:00.458236    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93\": container with ID starting with a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93 not found: ID does not exist" containerID="a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93"
Jul 28 02:01:00.458370 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.458266    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93"} err="failed to get container status \"a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93\": rpc error: code = NotFound desc = could not find container \"a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93\": container with ID starting with a9b410ee38558955ec8657f7205efbba97698ce7de98053a22446ce87b226f93 not found: ID does not exist"
Jul 28 02:01:00.458370 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.458287    2566 scope.go:117] "RemoveContainer" containerID="37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b"
Jul 28 02:01:00.467166 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.467137    2566 scope.go:117] "RemoveContainer" containerID="37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b"
Jul 28 02:01:00.467452 ip-10-0-131-236 kubenswrapper[2566]: E0728 02:01:00.467434    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b\": container with ID starting with 37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b not found: ID does not exist" containerID="37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b"
Jul 28 02:01:00.467516 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.467461    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b"} err="failed to get container status \"37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b\": rpc error: code = NotFound desc = could not find container \"37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b\": container with ID starting with 37c95059d0619b9938beda52d8ffe0b800c1e00c09a7bd79857dc5e73b8f0e5b not found: ID does not exist"
Jul 28 02:01:00.476319 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.476290    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"]
Jul 28 02:01:00.484338 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.484307    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-0337e8-85f7899544-mxft6"]
Jul 28 02:01:00.494033 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.494008    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"]
Jul 28 02:01:00.497954 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.497929    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-0337e8-764884fc94-2s4x9"]
Jul 28 02:01:00.561299 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.561222    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f720199-3725-4252-8280-a5122beb7590" path="/var/lib/kubelet/pods/2f720199-3725-4252-8280-a5122beb7590/volumes"
Jul 28 02:01:00.561546 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.561534    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="458fc273-7327-4971-9a99-022b941e37c7" path="/var/lib/kubelet/pods/458fc273-7327-4971-9a99-022b941e37c7/volumes"
Jul 28 02:01:00.561772 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.561761    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60407a9b-d80e-4f7d-aa93-1dbad04a51bd" path="/var/lib/kubelet/pods/60407a9b-d80e-4f7d-aa93-1dbad04a51bd/volumes"
Jul 28 02:01:00.562043 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:00.562031    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef226bd-348d-40a3-88f9-b8a124df15b1" path="/var/lib/kubelet/pods/cef226bd-348d-40a3-88f9-b8a124df15b1/volumes"
Jul 28 02:01:04.830453 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:04.830417    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 02:01:07.199998 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.199947    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"]
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200291    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f720199-3725-4252-8280-a5122beb7590" containerName="payload-pre-processing"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200303    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f720199-3725-4252-8280-a5122beb7590" containerName="payload-pre-processing"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200317    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef226bd-348d-40a3-88f9-b8a124df15b1" containerName="payload-processing"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200323    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef226bd-348d-40a3-88f9-b8a124df15b1" containerName="payload-processing"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200342    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458fc273-7327-4971-9a99-022b941e37c7" containerName="revoke-keys"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200349    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="458fc273-7327-4971-9a99-022b941e37c7" containerName="revoke-keys"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200396    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f720199-3725-4252-8280-a5122beb7590" containerName="payload-pre-processing"
Jul 28 02:01:07.200404 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200405    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef226bd-348d-40a3-88f9-b8a124df15b1" containerName="payload-processing"
Jul 28 02:01:07.200663 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.200413    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="458fc273-7327-4971-9a99-022b941e37c7" containerName="revoke-keys"
Jul 28 02:01:07.202580 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.202564    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:07.205142 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.205109    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-ck8gn\""
Jul 28 02:01:07.205279 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.205152    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 02:01:07.211189 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.211163    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"]
Jul 28 02:01:07.246076 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.246044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d7be3a-dfd5-4c0b-9316-beb34811fffb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj\" (UID: \"c2d7be3a-dfd5-4c0b-9316-beb34811fffb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:07.346711 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.346672    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d7be3a-dfd5-4c0b-9316-beb34811fffb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj\" (UID: \"c2d7be3a-dfd5-4c0b-9316-beb34811fffb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:07.347329 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.347311    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d7be3a-dfd5-4c0b-9316-beb34811fffb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj\" (UID: \"c2d7be3a-dfd5-4c0b-9316-beb34811fffb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:07.513627 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.513531    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:07.660296 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:07.660251    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"]
Jul 28 02:01:07.667749 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:01:07.667723    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2d7be3a_dfd5_4c0b_9316_beb34811fffb.slice/crio-2e42ad7acfeac26ebc91b74da67cf94b024462bcc80661897c654e12e10a40ae WatchSource:0}: Error finding container 2e42ad7acfeac26ebc91b74da67cf94b024462bcc80661897c654e12e10a40ae: Status 404 returned error can't find the container with id 2e42ad7acfeac26ebc91b74da67cf94b024462bcc80661897c654e12e10a40ae
Jul 28 02:01:08.494167 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:08.494125    2566 generic.go:356] "Generic (PLEG): container finished" podID="c2d7be3a-dfd5-4c0b-9316-beb34811fffb" containerID="6ace33e54036c5f64168519bfacba65718317878c477e690c320e936181c7ca6" exitCode=0
Jul 28 02:01:08.494569 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:08.494170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj" event={"ID":"c2d7be3a-dfd5-4c0b-9316-beb34811fffb","Type":"ContainerDied","Data":"6ace33e54036c5f64168519bfacba65718317878c477e690c320e936181c7ca6"}
Jul 28 02:01:08.494569 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:08.494233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj" event={"ID":"c2d7be3a-dfd5-4c0b-9316-beb34811fffb","Type":"ContainerStarted","Data":"2e42ad7acfeac26ebc91b74da67cf94b024462bcc80661897c654e12e10a40ae"}
Jul 28 02:01:09.633120 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:09.633097    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:09.667447 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:09.667417    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d7be3a-dfd5-4c0b-9316-beb34811fffb-maas-api-service-ca\") pod \"c2d7be3a-dfd5-4c0b-9316-beb34811fffb\" (UID: \"c2d7be3a-dfd5-4c0b-9316-beb34811fffb\") "
Jul 28 02:01:09.667823 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:09.667797    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2d7be3a-dfd5-4c0b-9316-beb34811fffb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2d7be3a-dfd5-4c0b-9316-beb34811fffb" (UID: "c2d7be3a-dfd5-4c0b-9316-beb34811fffb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:09.768281 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:09.768198    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2d7be3a-dfd5-4c0b-9316-beb34811fffb-maas-api-service-ca\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:10.506157 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:10.506132    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"
Jul 28 02:01:10.506336 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:10.506130    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj" event={"ID":"c2d7be3a-dfd5-4c0b-9316-beb34811fffb","Type":"ContainerDied","Data":"2e42ad7acfeac26ebc91b74da67cf94b024462bcc80661897c654e12e10a40ae"}
Jul 28 02:01:10.506336 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:10.506254    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e42ad7acfeac26ebc91b74da67cf94b024462bcc80661897c654e12e10a40ae"
Jul 28 02:01:17.637699 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.637651    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"]
Jul 28 02:01:17.639319 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.639295    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-44b320-29753400-rx65t"]
Jul 28 02:01:17.668985 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.668944    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"]
Jul 28 02:01:17.669187 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.669163    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt" podUID="5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" containerName="payload-processing" containerID="cri-o://256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8" gracePeriod=30
Jul 28 02:01:17.680087 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.673719    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"]
Jul 28 02:01:17.683322 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.681491    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw" podUID="261c1507-f81a-47bd-b03f-98ac08fbf5e4" containerName="payload-pre-processing" containerID="cri-o://819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a" gracePeriod=30
Jul 28 02:01:17.826693 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.826643    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"]
Jul 28 02:01:17.829608 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.829581    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-44b320-pnrzj"]
Jul 28 02:01:17.965812 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.965791    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 02:01:17.968919 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:17.968900    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 02:01:18.040699 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.040663    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gcsk\" (UniqueName: \"kubernetes.io/projected/261c1507-f81a-47bd-b03f-98ac08fbf5e4-kube-api-access-9gcsk\") pod \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") "
Jul 28 02:01:18.040878 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.040715    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c1507-f81a-47bd-b03f-98ac08fbf5e4-plugins-config-volume\") pod \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\" (UID: \"261c1507-f81a-47bd-b03f-98ac08fbf5e4\") "
Jul 28 02:01:18.040878 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.040743    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hngp\" (UniqueName: \"kubernetes.io/projected/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-kube-api-access-4hngp\") pod \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") "
Jul 28 02:01:18.040878 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.040772    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-plugins-config-volume\") pod \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\" (UID: \"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64\") "
Jul 28 02:01:18.041234 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.041205    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261c1507-f81a-47bd-b03f-98ac08fbf5e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "261c1507-f81a-47bd-b03f-98ac08fbf5e4" (UID: "261c1507-f81a-47bd-b03f-98ac08fbf5e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:18.041365 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.041232    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" (UID: "5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 02:01:18.043052 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.043033    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-kube-api-access-4hngp" (OuterVolumeSpecName: "kube-api-access-4hngp") pod "5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" (UID: "5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64"). InnerVolumeSpecName "kube-api-access-4hngp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:01:18.043144 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.043121    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261c1507-f81a-47bd-b03f-98ac08fbf5e4-kube-api-access-9gcsk" (OuterVolumeSpecName: "kube-api-access-9gcsk") pod "261c1507-f81a-47bd-b03f-98ac08fbf5e4" (UID: "261c1507-f81a-47bd-b03f-98ac08fbf5e4"). InnerVolumeSpecName "kube-api-access-9gcsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 02:01:18.141896 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.141859    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gcsk\" (UniqueName: \"kubernetes.io/projected/261c1507-f81a-47bd-b03f-98ac08fbf5e4-kube-api-access-9gcsk\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.141896 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.141890    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261c1507-f81a-47bd-b03f-98ac08fbf5e4-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.141896 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.141900    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hngp\" (UniqueName: \"kubernetes.io/projected/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-kube-api-access-4hngp\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.142166 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.141911    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64-plugins-config-volume\") on node \"ip-10-0-131-236.ec2.internal\" DevicePath \"\""
Jul 28 02:01:18.545953 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.545917    2566 generic.go:356] "Generic (PLEG): container finished" podID="5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" containerID="256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8" exitCode=0
Jul 28 02:01:18.546179 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.545982    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"
Jul 28 02:01:18.546179 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.546001    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt" event={"ID":"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64","Type":"ContainerDied","Data":"256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8"}
Jul 28 02:01:18.546179 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.546049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt" event={"ID":"5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64","Type":"ContainerDied","Data":"0d9f82b4c127493a051c9dab696249c1506cb056d9df293a27f5670c5b1cfdba"}
Jul 28 02:01:18.546179 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.546070    2566 scope.go:117] "RemoveContainer" containerID="256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8"
Jul 28 02:01:18.547413 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.547383    2566 generic.go:356] "Generic (PLEG): container finished" podID="261c1507-f81a-47bd-b03f-98ac08fbf5e4" containerID="819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a" exitCode=0
Jul 28 02:01:18.547552 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.547431    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"
Jul 28 02:01:18.547552 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.547486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw" event={"ID":"261c1507-f81a-47bd-b03f-98ac08fbf5e4","Type":"ContainerDied","Data":"819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a"}
Jul 28 02:01:18.547552 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.547525    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw" event={"ID":"261c1507-f81a-47bd-b03f-98ac08fbf5e4","Type":"ContainerDied","Data":"1b2865e0acde47a8c09e3af0ac14eb8330ea16d852dac9ff222db51734ca31af"}
Jul 28 02:01:18.556095 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.555933    2566 scope.go:117] "RemoveContainer" containerID="256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8"
Jul 28 02:01:18.556331 ip-10-0-131-236 kubenswrapper[2566]: E0728 02:01:18.556303    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8\": container with ID starting with 256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8 not found: ID does not exist" containerID="256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8"
Jul 28 02:01:18.556406 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.556342    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8"} err="failed to get container status \"256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8\": rpc error: code = NotFound desc = could not find container \"256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8\": container with ID starting with 256535b5b961fde63a8fb550c69349a0c59875da9c9fa329bb6317686c3c84f8 not found: ID does not exist"
Jul 28 02:01:18.556406 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.556358    2566 scope.go:117] "RemoveContainer" containerID="819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a"
Jul 28 02:01:18.561549 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.561524    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c85757b-a0a2-4909-9e7b-032488a92c1e" path="/var/lib/kubelet/pods/3c85757b-a0a2-4909-9e7b-032488a92c1e/volumes"
Jul 28 02:01:18.561800 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.561787    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2d7be3a-dfd5-4c0b-9316-beb34811fffb" path="/var/lib/kubelet/pods/c2d7be3a-dfd5-4c0b-9316-beb34811fffb/volumes"
Jul 28 02:01:18.565049 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.565034    2566 scope.go:117] "RemoveContainer" containerID="819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a"
Jul 28 02:01:18.565288 ip-10-0-131-236 kubenswrapper[2566]: E0728 02:01:18.565271    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a\": container with ID starting with 819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a not found: ID does not exist" containerID="819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a"
Jul 28 02:01:18.565347 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.565295    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a"} err="failed to get container status \"819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a\": rpc error: code = NotFound desc = could not find container \"819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a\": container with ID starting with 819927e6a94e90926da70e1b19c0e42a051ce02b2e41fc76eba706ba41933d5a not found: ID does not exist"
Jul 28 02:01:18.571173 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.571151    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"]
Jul 28 02:01:18.584716 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.580912    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-44b320-7f99fd4bcb-d5tmw"]
Jul 28 02:01:18.593107 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.593083    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"]
Jul 28 02:01:18.601223 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:18.601194    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-44b320-5795cb9c66-8nlmt"]
Jul 28 02:01:20.333172 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:20.333134    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-cpd2s"]
Jul 28 02:01:20.561312 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:20.561281    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261c1507-f81a-47bd-b03f-98ac08fbf5e4" path="/var/lib/kubelet/pods/261c1507-f81a-47bd-b03f-98ac08fbf5e4/volumes"
Jul 28 02:01:20.561621 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:20.561608    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" path="/var/lib/kubelet/pods/5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64/volumes"
Jul 28 02:01:31.051905 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:31.051874    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv_c7832476-bdc4-48b4-887c-8215a257964a/copy-manifests/0.log"
Jul 28 02:01:31.059201 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:31.059178    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv_c7832476-bdc4-48b4-887c-8215a257964a/manager/0.log"
Jul 28 02:01:31.614541 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:31.614513    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753385-k4jht_a5f8fd4e-9383-401b-938b-fc42839573aa/cleanup/0.log"
Jul 28 02:01:31.719400 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:31.719373    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753400-55dn7_f096d179-9fb9-467b-b617-19ff05bc46fa/cleanup/0.log"
Jul 28 02:01:33.716821 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:33.716791    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cpd2s_d6934dbe-c0be-48e6-918e-b35819dd9b61/limitador/0.log"
Jul 28 02:01:34.292505 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:34.292472    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8p648_1c6fc6dc-d5b9-413e-b2d3-a4d17848f224/discovery/0.log"
Jul 28 02:01:34.983797 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:34.983759    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b598fd8c5-s7dmr_2e916bb7-6c9e-40e5-9ca7-a7d242bd9384/router/0.log"
Jul 28 02:01:35.431312 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:35.431285    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mptxp_a060e5f2-30a7-4fcc-8475-624bf962a250/main/0.log"
Jul 28 02:01:45.304496 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:45.304461    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hdkfq_2c86d027-c224-4ab8-8b80-36d85be6e41b/global-pull-secret-syncer/0.log"
Jul 28 02:01:45.462879 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:45.462849    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v28qs_e1388316-7ecb-428d-8e92-a983372bdf5b/konnectivity-agent/0.log"
Jul 28 02:01:45.485442 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:45.485417    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-236.ec2.internal_b3d4f2c0da36356c1d7f9784538048ba/haproxy/0.log"
Jul 28 02:01:51.867264 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:51.867229    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-cpd2s_d6934dbe-c0be-48e6-918e-b35819dd9b61/limitador/0.log"
Jul 28 02:01:53.883800 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:53.883771    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rz6tq_5b320f00-bbc1-454e-b303-e1113262789c/kube-state-metrics/0.log"
Jul 28 02:01:53.904491 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:53.904461    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rz6tq_5b320f00-bbc1-454e-b303-e1113262789c/kube-rbac-proxy-main/0.log"
Jul 28 02:01:53.937306 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:53.937273    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-rz6tq_5b320f00-bbc1-454e-b303-e1113262789c/kube-rbac-proxy-self/0.log"
Jul 28 02:01:53.990582 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:53.990549    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-62sc8_fd70dd90-a4be-46c5-9ab3-1381a361d9c7/monitoring-plugin/0.log"
Jul 28 02:01:54.204877 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:54.204792    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nvvjp_ae5e1a85-7436-492a-8776-938f82ff4257/node-exporter/0.log"
Jul 28 02:01:54.225002 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:54.224958    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nvvjp_ae5e1a85-7436-492a-8776-938f82ff4257/kube-rbac-proxy/0.log"
Jul 28 02:01:54.244454 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:54.244431    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nvvjp_ae5e1a85-7436-492a-8776-938f82ff4257/init-textfile/0.log"
Jul 28 02:01:54.650398 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:54.650365    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-r896j_3adb5bd3-f307-42ae-bd22-3ca0724ea19c/prometheus-operator/0.log"
Jul 28 02:01:54.671267 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:54.671238    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-r896j_3adb5bd3-f307-42ae-bd22-3ca0724ea19c/kube-rbac-proxy/0.log"
Jul 28 02:01:54.701673 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:54.701646    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-6qkll_536fa5ca-bba3-4d3e-9c67-0ce5a05d0ac7/prometheus-operator-admission-webhook/0.log"
Jul 28 02:01:56.562361 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562332    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"]
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562619    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261c1507-f81a-47bd-b03f-98ac08fbf5e4" containerName="payload-pre-processing"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562630    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="261c1507-f81a-47bd-b03f-98ac08fbf5e4" containerName="payload-pre-processing"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562651    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" containerName="payload-processing"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562656    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" containerName="payload-processing"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562667    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2d7be3a-dfd5-4c0b-9316-beb34811fffb" containerName="revoke-keys"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562672    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2d7be3a-dfd5-4c0b-9316-beb34811fffb" containerName="revoke-keys"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562723    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2d7be3a-dfd5-4c0b-9316-beb34811fffb" containerName="revoke-keys"
Jul 28 02:01:56.562729 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562732    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="261c1507-f81a-47bd-b03f-98ac08fbf5e4" containerName="payload-pre-processing"
Jul 28 02:01:56.563010 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.562738    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f550aa8-c619-4dd0-ba2d-cd75e2d8fa64" containerName="payload-processing"
Jul 28 02:01:56.565077 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.565062    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.567643 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.567622    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pzk5p\"/\"kube-root-ca.crt\""
Jul 28 02:01:56.567776 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.567623    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pzk5p\"/\"openshift-service-ca.crt\""
Jul 28 02:01:56.568325 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.568306    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pzk5p\"/\"default-dockercfg-wjwmd\""
Jul 28 02:01:56.575776 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.575753    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"]
Jul 28 02:01:56.674458 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.674422    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gwfk\" (UniqueName: \"kubernetes.io/projected/a6057e2e-7dbf-453a-8ad1-c984183f945f-kube-api-access-2gwfk\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.674458 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.674459    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-proc\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.674680 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.674533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-podres\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.674680 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.674586    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-sys\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.674680 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.674607    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-lib-modules\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776064 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776026    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-sys\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776064 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-lib-modules\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776117    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gwfk\" (UniqueName: \"kubernetes.io/projected/a6057e2e-7dbf-453a-8ad1-c984183f945f-kube-api-access-2gwfk\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-proc\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-sys\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776218    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-proc\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-podres\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776241    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-lib-modules\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.776390 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.776301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6057e2e-7dbf-453a-8ad1-c984183f945f-podres\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.785161 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.785138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gwfk\" (UniqueName: \"kubernetes.io/projected/a6057e2e-7dbf-453a-8ad1-c984183f945f-kube-api-access-2gwfk\") pod \"perf-node-gather-daemonset-mqn6c\" (UID: \"a6057e2e-7dbf-453a-8ad1-c984183f945f\") " pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:56.875154 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:56.875120    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:57.004638 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:57.004605    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"]
Jul 28 02:01:57.007649 ip-10-0-131-236 kubenswrapper[2566]: W0728 02:01:57.007609    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda6057e2e_7dbf_453a_8ad1_c984183f945f.slice/crio-8fe5c285fbfd16625e70fbc2208a7a4cfd54dadb6147abf7ee22bfa90467f865 WatchSource:0}: Error finding container 8fe5c285fbfd16625e70fbc2208a7a4cfd54dadb6147abf7ee22bfa90467f865: Status 404 returned error can't find the container with id 8fe5c285fbfd16625e70fbc2208a7a4cfd54dadb6147abf7ee22bfa90467f865
Jul 28 02:01:57.741355 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:57.741310    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c" event={"ID":"a6057e2e-7dbf-453a-8ad1-c984183f945f","Type":"ContainerStarted","Data":"5d3a2a58475731b1670264359079413c2909dd15ee34514ee69ca76108a7bbfb"}
Jul 28 02:01:57.741794 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:57.741371    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c" event={"ID":"a6057e2e-7dbf-453a-8ad1-c984183f945f","Type":"ContainerStarted","Data":"8fe5c285fbfd16625e70fbc2208a7a4cfd54dadb6147abf7ee22bfa90467f865"}
Jul 28 02:01:57.741794 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:57.741491    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:01:57.757023 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:57.756961    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c" podStartSLOduration=1.756944576 podStartE2EDuration="1.756944576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 02:01:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 02:01:57.756105081 +0000 UTC m=+3453.767343691" watchObservedRunningTime="2026-07-28 02:01:57.756944576 +0000 UTC m=+3453.768183184"
Jul 28 02:01:57.843475 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:57.843446    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-zp9pj_cb54c5d5-d5ac-4456-95c6-c1d8de186d47/volume-data-source-validator/0.log"
Jul 28 02:01:58.966521 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:58.966491    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvg8g_d13e3477-1101-406b-a2a6-4fcd0e206eec/dns/0.log"
Jul 28 02:01:58.986985 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:58.986941    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvg8g_d13e3477-1101-406b-a2a6-4fcd0e206eec/kube-rbac-proxy/0.log"
Jul 28 02:01:59.008770 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:59.008744    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-58nwp_db9c66a8-6859-4e59-a9ec-938b424484f0/dns-node-resolver/0.log"
Jul 28 02:01:59.679557 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:01:59.679519    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-n6jdz_8443e39b-0f3d-423c-ad49-0ab06a0140e8/node-ca/0.log"
Jul 28 02:02:00.782721 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:00.782687    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8p648_1c6fc6dc-d5b9-413e-b2d3-a4d17848f224/discovery/0.log"
Jul 28 02:02:00.991798 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:00.991768    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b598fd8c5-s7dmr_2e916bb7-6c9e-40e5-9ca7-a7d242bd9384/router/0.log"
Jul 28 02:02:01.830537 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:01.830505    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zhqzt_c966ed32-b941-4d8a-a3aa-449f757957b6/serve-healthcheck-canary/0.log"
Jul 28 02:02:02.295944 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:02.295907    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-m68h5_253a805e-0ba8-4c8d-ab32-776f7eeb93ab/insights-operator/0.log"
Jul 28 02:02:02.296698 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:02.296678    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-m68h5_253a805e-0ba8-4c8d-ab32-776f7eeb93ab/insights-operator/1.log"
Jul 28 02:02:02.319134 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:02.319102    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cd6hz_197c23c3-1540-404a-91ff-5a62e4fa333f/kube-rbac-proxy/0.log"
Jul 28 02:02:02.346534 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:02.346499    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cd6hz_197c23c3-1540-404a-91ff-5a62e4fa333f/exporter/0.log"
Jul 28 02:02:02.369808 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:02.369779    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cd6hz_197c23c3-1540-404a-91ff-5a62e4fa333f/extractor/0.log"
Jul 28 02:02:03.757063 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:03.757030    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pzk5p/perf-node-gather-daemonset-mqn6c"
Jul 28 02:02:05.477940 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:05.477885    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv_c7832476-bdc4-48b4-887c-8215a257964a/manager/0.log"
Jul 28 02:02:05.499227 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:05.499201    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f5bf98dfb-vmjtv_c7832476-bdc4-48b4-887c-8215a257964a/copy-manifests/0.log"
Jul 28 02:02:05.988453 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:05.988362    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8p648_1c6fc6dc-d5b9-413e-b2d3-a4d17848f224/discovery/0.log"
Jul 28 02:02:06.141841 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:06.141810    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b598fd8c5-s7dmr_2e916bb7-6c9e-40e5-9ca7-a7d242bd9384/router/0.log"
Jul 28 02:02:06.922356 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:06.922326    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d99f9899c-mbvlv_506f057a-5674-4078-a16f-1022149fb34e/manager/0.log"
Jul 28 02:02:06.977511 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:06.977432    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8kvrn_f5597c68-6e5f-48f4-9b4e-836b9aef3bcc/openshift-lws-operator/0.log"
Jul 28 02:02:07.751274 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:07.751246    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-8p648_1c6fc6dc-d5b9-413e-b2d3-a4d17848f224/discovery/0.log"
Jul 28 02:02:07.942611 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:07.942584    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5b598fd8c5-s7dmr_2e916bb7-6c9e-40e5-9ca7-a7d242bd9384/router/0.log"
Jul 28 02:02:12.236347 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:12.236312    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-6764s_106ccc7e-fbc5-4b90-8eee-829b1146ed20/kube-storage-version-migrator-operator/1.log"
Jul 28 02:02:12.237130 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:12.237114    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-6764s_106ccc7e-fbc5-4b90-8eee-829b1146ed20/kube-storage-version-migrator-operator/0.log"
Jul 28 02:02:13.294181 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.294149    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6ln99_29001342-cb93-42de-a039-2e4558104d8f/kube-multus/0.log"
Jul 28 02:02:13.525536 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.525507    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/kube-multus-additional-cni-plugins/0.log"
Jul 28 02:02:13.544381 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.544319    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/egress-router-binary-copy/0.log"
Jul 28 02:02:13.564452 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.564428    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/cni-plugins/0.log"
Jul 28 02:02:13.584866 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.584839    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/bond-cni-plugin/0.log"
Jul 28 02:02:13.605292 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.605271    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/routeoverride-cni/0.log"
Jul 28 02:02:13.625083 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.625062    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/whereabouts-cni-bincopy/0.log"
Jul 28 02:02:13.644901 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.644879    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rh6sg_0fb4f15a-ca09-41c6-86dc-d99584578a2e/whereabouts-cni/0.log"
Jul 28 02:02:13.882221 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.882195    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fnd2j_b15ae3bf-ee89-4a3f-9ae1-565c670d2b31/network-metrics-daemon/0.log"
Jul 28 02:02:13.901922 ip-10-0-131-236 kubenswrapper[2566]: I0728 02:02:13.901897    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fnd2j_b15ae3bf-ee89-4a3f-9ae1-565c670d2b31/kube-rbac-proxy/0.log"